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8th  Annual  Systems  Engineering  Conference 
“Focusing  on  Mission  Areasy  Net-Centric  Operations 
and  Supportability  of  Defense  Systems 99 

San  Diego,  CA 

24-27  October  2005 


Agenda 


Tuesday.  25  October  2005 


Open  Remarks:  by  Mr.  Bob  Rassa,  Director,  Systems  Supportability,  Raytheon;  Chair,  Systems  Engineering  Division,  NDIA 
Keynote  Address:  by  Mr.  John  Landon,  Deputy  Assistant  Secretary  of  Defense  (Nil),  C3ISR  &  IT  Acquisition 

Plenary  Session  -  Revitalization  of  Systems  Engineering  Within  DoD: 

•  State  of  Systems  Engineering  within  DoDs,  Mr.  Mark  D.  Schaeffer,  Deputy  Director,  Systems  Engineering,  OUSD  (AT&L) 

•  USAF  Systems  Engineering  Initiatives,  Mr.  Terry  Jaggers,  SAF/AQR  (Science  &  Technology  &  Engineering) 

•  System  Engineering  Re-vitalization  within  DoN  Status,  Mr.  Carl  Siel,  ASN(RDA)  Chief  Engineer 

•  Army  SE  Overview,  Mr.  Douglas  K.  Wiltsie,  Assistant  Deputy,  Acquisition  and  Systems  Management,  Office  of  the  Assistant  Secretary  of  the  Army, 
Acquisition  Logistics  and  Technology 

•  “Implementation  of  ESE/A”,  Mr.  Kelly  A.  Miller,  NSA/CSS  CSE 

Luncheon  Keynote  Speaker:  by  Mr.  Gregory  Shelton,  Corporate  Vice  President,  Engineering,  Technology,  Manufacturing  and  Quality,  Raytheon  Company 
Tracks  1  &  2  -  Systems  Engineering  Effectiveness: 

•  Technical  Planning  for  Acquisition  Programs:  An  OSD  Perspective,  Col  Warren  Anderson,  OUSD  (AT&L)  Defense  Systems 

•  Implementation  of  Policy  Requiring  Systems  Engineering  Plans  for  Air  Force  Programs  -  Results  and  Implications,  Mr.  Kevin  Kemper,  Air  Force  Materiel 
Command 

•  Systems  Engineering  Revitalization  at  SPAWAR  Systems  Center  Charleston,  Mr.  Michael  T.  Kutch,  Jr.,  SPAWAR  Systems  Center 

•  Systems  Engineering  for  Software  Assurance,  Ms.  Kristen  Baldwin,  OUSD  (AT&L)  Defense  Systems 

•  Revitalization  of  Systems  Engineering:  Past,  Present  and  Future,  Ms.  Karen  B.  Bausman,  Air  Force  Center  for  Systems  Engineering 

•  Enabling  Technology  Readiness  Assessments  (TRAs)  with  Systems  Engineering,  Dr.  Jay  Mandelbaum,  Institute  for  Defense  Analyses 

•  A  Taxonomy  of  Operational  Risks,  Mr.  Brian  Gallagher,  Software  Engineering  Institute 

•  A  Method  for  Reasoning  About  an  Acquisition  Strategy,  Mr.  Joseph  Elm,  Software  Engineerin  Institute 

•  WBS-Based  Approach  to  Understanding  and  Predicting  Program  Risk,  Bruce  M.  Heim,  DCMA,  Boeing  Long  Beach 

•  Program  Support:  Perspectives  on  Technical  Planning  and  Execution,  Mr.  Dave  Castellano,  OUSD  (AT&L)  Systems  Engineering 

Track  3  -  Test  &  Evaluation  in  Systems  Engineering: 

•  Interweaving  Test  and  Evaluation  Throughout  the  Systems  Engineering  Process  -  Presentation  and  Paper,  Mr.  Josh  Tribble,  AVW  Technologies 
Track  4  -  Net  Centric  Operations: 

•  Net-Centricity  &  Net-Ready  -  Beyond  Technical  Interoperability  &  C4ISR,  Mr.  Jack  Zavin,  ASD(NII),  DoD  CIO/A&I  Directorate 

•  A  Strategy  for  Managing  Development  and  Certification  of  Net-Centric  Services  within  the  Global  Information  Grid,  Mr.  Bernal  Allen,  DISA,  GE  4 

•  Next  Generation  Enterprise  Information  Management  Appliances,  Mr.  Michael  Lindow,  The  MITRE  Corp. 

Track  5  -  Logistics: 

•  Logistics  Transforming:  Achieving  Knowledge-Enabled  Logistics,  Mr.  Jerry  Beck,  OSD  Office  of  ADUSD(LPP) 

•  Condition  Based  Logistics,  Mr.  Ron  Wagner,  CoBaLt  Technology 

•  System  Supportability  and  Life  Cycle  Product  Support:  A  Systems  Perspective,  Dinesh  Verma,  Stevens  Institute  of  Technolog 

•  The  Management  of  Logistics  in  Large  Scale  Inventory  Systems  to  Support  Weapon  System  Maintenance,  Mr.  Eugene  A.  Beardslee,  SAIC 

Track  7  -  Systems  Safety: 
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•  System  Safety  in  Systems  Engineering  DAU  Continuous  Learning  Module,  Ms.  Amanda  Zarecky,  Booz  Allen  Hamilton 

•  Enabling  System  Safety  Through  Technical  Excellence,  Col  Warren  Anderson,  OUSD  (AT&L)  Defense  Systems 

•  Applying  CMMI  to  System  Safety,  Mr.  Tom  Pfitzer,  APT  Research,  Inc. 

•  System  Safety  Engineering:  An  Overview  for  Engineers  and  Managers,  Mr.  Pat  L.  Clemens,  APT  Research,  Inc. 

•  Using  MIL-STD-882D  to  Integrate  ESOH  into  SE,  Mr.  Sherman  G.  Forbes,  USAF  -  SAF/AQRE 

Track  8  -  Software  Supportabilitv: 

•  The  Proper  Specification  of  Requirements,  Mr.  A1  Florence,  The  MITRE  Corporation 

•  C-17  Software  Development  Process,  John  R.  Allen,  The  Boeing  Company  4 

•  Successful  Verification  and  Validation  Based  on  the  CMMI  Model,  Mr.  Tim  Olson,  Quality  Improvement  Consultants,  Inc. 

•  “Automated  Software  Testing  Increases  Test  Quality  and  Coverage  Resulting  in  Improved  Software  Reliability.”,  Mr.  Frank  Salvatore,  High  Performance 
Technologies,  Inc. 

•  Software  Supportability:  A  Software  Engineering  Perspective,  Ms.  Stephany  Bellomo,  SAIC 


Wednesday.  26  October  2005 


Tracks  1,  2  &  3  -  Systems  Engineering  Effectiveness: 

•  Decision  Analysis  and  Resolution,  Mr  Robert  Trifiletti,  Jr.,  US  Army  ARDEC 

•  Defining  System  Development  Lifecycles  to  Plan  and  Manage  Projects  Effectively,  Mr.  Bruce  A.  Boyd,  The  Boeing  Company 

•  Systems  Engineering,  Program  Management  conjoined  Disciplines  over  the  Project  Life  Cycle,  Mr.  William  Lyders,  ASSETT,  Inc. 

•  Tailoring  USAF  Systems  Engineering  for  the  Life  Cycle:  One  Shape,  Multiple  Dimensions,  Mr.  Jeff  Loren,  MTC  Technologies,  Inc.  (SAF/AQRE) 

•  Architecture-Based  Systems  Engineering  and  Integration,  Dr.  Rick  Habayeb,  Virginia  Polytechnic  Institute  &  State  University 

•  A  Complementary  Approach  to  Enterprise  Systems  Engineering,  Dr.  Brian  White,  The  MITRE  Corporation 

•  Implementing  Systems  Engineering  Processes  to  Balance  Cost  and  Technical  Performance,  Dr.  Mary  Anne  Herndon,  Transdyne  Corporation 

•  Program  Support:  Perspectives  on  Technical  Planning  and  Execution,  Mr.  Dave  Castellano,  OUSD  (AT&L)  Systems  Engineering 

•  Application  of  Risk  Management  in  a  Net-Centric  Environment,  Ms.  Rebecca  M.  Cowen-Hirsch,  DISA 

•  “Requirements  Management  Tips  and  Tricks”,  Mr.  Frank  Salvatore,  High  Performance  Technologies,  Inc. 

•  Engineering  and  Implementing  Raytheon  Missile  Systems  Engineering  Design  to  Cost  Metric  -  Presentation  and  Paper,  Mr.  Edward  Casey,  Raytheon  Missile 
Systems 

•  System  Engineering  Metrics,  Mr.  James  Miller,  Air  Foce  Materiel  Command 

•  Technical  Performance  Measures,  Mr.  Jim  Oakes,  BAE  Systems 

•  TurboTax®  for  Systems  EngineerinTurboTax®  for  Systems  Engineering,  Michael  T.  Kutch,  Jr.,  SPAWAR 

•  A  Practical  Application  of  A  Practical  Application  of  the  Non- Advocate  Review,  Mr.  Bruce  Nishime,  The  Boeing  Company 

•  Systems  Engineering  and  the  Software  Laws  of  Thermodynamics,  Dr.  Thomas  F.  Christian  Jr.,  402  SMXG 

•  Unmanned  Aerial  Vehicle  Survivability  Influence  on  System  Life  Cycle  Cost,  Mr.  Chuck  Pedriani,  SURVICE  Engineering 

•  Effective  SE  Metrics  Tailored  to  the  Acquisition  Life  Cycle,  Ms.  Laura  Trioilia,  US  Army  ARDEC 

•  Innovative  Procurement  Strategies,  Mr.  David  Eiband,  Defense  Acquisition  University 

•  Next  Generation  Combat  Systems  -  An  Overview  of  Key  Development  Concepts,  Mr.  Matthew  Montoya,  The  JHU  Applied  Physics  Laboratory  Mr.  Edward 
Casey,  Raytheon  Missile  Systems 

•  Converting  High-Level  Systems  Engineering  Policy  to  a  Workable  Program,  Mr.  James  Miller,  Air  Force  Materiel  Command 

•  AFRL  Systems  Engineering  Initiative  -  Risk  Managment  for  Science  and  Technology,  Mr.  William  Nolte,  USAF-AFRL 

•  System  Engineered  Research  and  Development  Magement,  Dr.  Steven  Ligon,  SAIC 

•  The  Return  of  Discipline,  Ms.  Jacqueline  Townsend,  Air  Force  Materiel  Command 

Track  4  -  Net  Centric  Operations: 

•  Testing  Net-Centric  Systems  of  Systems:  Applying  Lessons  Learned  from  Distributed  Simulation,  Mr.  Doug  Flournoy,  The  MITRE  Corp. 

•  A  Multi-Mission  Network  Centric  Warfare  Platform,  Peder  Jungck,  CloudSheild  Technologies 

•  Challenges  Challenges  in  Development  of  System  of  Systems  (SoS)  Architectures  in  a  Net  Centric  Environment,  Dr.  Abraham  Meilich,  Lockheed  Martin 

•  Matrix  Mapping  Tool  (MMT),  Dr.  Judith  Dahmann,  AT&L/DS  MITRE 

Track  5  -  Logistics: 

•  Defense  Logistics  as  Chaos  Theory,  Mr.  John  Sells,  Tobyhanna  Army  Depot 

•  Process  for  Evaluating  LogisticProcess  for  Evaluating  Logistics  Readiness  Levels  (LRLs)  for  Acquisition  Systems,  Ms.  Elizabeth  Broadus,  Booz  Allen 
Hamilton,  Inc. 

•  The  Management  of  Logistics  in  Large  Scale  Inventory  Systems  to  Support  Weapon  System  Maintenance,  Mr.  Eugene  A.  Beardslee,  SAIC 

•  System  of  Systems  Analysis  of  Future  Combat  Systems  Sustainment  Requirements,  Mr.  Ivan  W.  Wolnek,  The  Boeing  Company 

•  Readiness  &  Supportability  Program  Readiness  &  Supportability  Programs,  Mr.  Robert  M.  Cranwell,  Sandia  National  Laboratories  (SNL) 

•  Data  Management  in  a  Performance  Based  Logistics  Environment,  Denise  Duncan,  LMI 

Track  5  -  Best  Practices  &  Standardization: 

•  CMMI  for  Services,  Mr.  Juan  Ceva,  Raytheon  Company 

•  Out  of  the  Ordinary:  Finding  Hidden  Threats  by  Analyzing  Unusual  Behavior,  Mr.  John  Hollywood,  RAND 
Track  6  -  Modeling  &  Simulation: 

•  Improving  M&S  Support  to  Acquisition:  A  Progress  Report  on  Development  of  the  Acquisition  M&S  Master  Plan,  Mr.  Jim  Hollenbach,  Simulation  Strategies, 
Inc. 
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•  Next  Generation  Manufacturing  Technology  Initiative  and  the  Model  -  Based  Enterprise,  Mr.  Richard  Neal  -  IMTI 

•  Problem  Space  Modeling:  A  Dynamic  Future  for  Requirements  Analysis,  Mr.  Jeffrey  O.  Grady,  JOG  System  Engineering,  Inc. 

•  Systems  Modeling  Language  Systems  Modeling  Language  (SysML)  Overview  &  Update,  Rick  Steiner,  Raytheon  Company 

•  Data  Management  Support  for  Modeling  and  Simulation,  Mr.  Denise  Duncan,  LMI 

•  Digital  Data  Management  an  Update,  Ms.  Cynthia  C.  Hauer,  Millennium  Data  Management,  Inc. 

•  The  Use  of  Simulation  in  the  Management  of  Logistics  in  Large  Scale  Inventory  Systems  to  Support  Weapon  System  Maintenance,  Mr.  Eugene  A.  Beardslee, 
SAIC 

Track  7  -  System  Safety: 

•  Mission  Sustainment  Through  Environment,  Safety,  and  Occupational  Health  (ESOH)  Risk  Management,  Ms.  Trish  Huheey,  ODUSD  (I&E) 

•  Lessons  Learned  with  the  Application  of  MIL-STD-882D  at  the  Weapon  System  Explosives  Safety  Review  Board,  Ms.  Mary  Ellen  Caro,  Ordnance  Safety  & 
Security  Activity 

•  Industry  Perspectives  and  Identified  Barriers  to  the  Use  of  MIL-STD-882D  for  Integrating  ESOH  Considerations  into  Systems,  Mr.  Jon  Derickson,  BAE 
Systems 

•  System  Safety  in  Systems  Engineering  Process,  Dr.  Ray  C.  Terry,  SURVICE  Engineering  Company 

•  Enabling  Army  Level  Risk  Mitigation,  Mr.  Bill  Edmonds,  US  Army  Combat  Readiness  Center 

•  Evolution  of  MIL-STD-882E,  Mr.  Robert  McAllister,  US  Air  Force  Materiel  Command 

•  Integrating  MIL- STD-8 82  System  Safety  Products  into  the  Concurrent  Engineering  Approach  to  System  Design,  Build,  Test,  and  Delivery  of  Submarine 
Systems  At  Electric  Boat,  Mr.  Ricky  Milnarik,  General  Dynamics 

Track  8  -  Legacy  Systems  Sustainment: 

•  Sustaining  Software-Intensive  Systems  -  A  Conundrum,  Ms.  Mary  Ann  Lapham,  Carnegie  Mellon  Software  Engineering  Institute 

•  Algorithm  Description  Documentation  and  Validation  Process,  Mr.  Mike  Bailey,  Raytheon  Company 

•  ATSRAC:  Background,  Results  and  Future  Impact  on  the  Aviation  Industry,  Mr.  Kent  V.  Hollinger,  The  MITRE  Corp. 

•  Jammer  Integration  Roadmap,  Mr.  Adam  McCorkle,  GTRI 

•  Open  Systems  Architecture  (OSA)  and  Standard  Interfaces  as  Mission  Capability  Enablers,  William  H.  Mish,  Jr.,  AMSEC 

•  Naval  Air  Systems  Command  Integrated  In-Service  Reliability  Program  (IISRP),  Mr.  Les  Wetherington,  Integrated  In-Service  Reliability  Program  (IISRP) 
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rant 


Sponsored  by  the 
National  Defense 
Industrial  Association, 
with  Technical  Co 'Sponsorship  by 
IEEE  AES,  IEEE  Systems  Council 
end  ENCOSE 
Supported  by 

Office  of  Under  Secretary  of 
Defense,  Acquisition  Technology  St 
LogiSUos,  Defense  Systems, 
Director,  Systems  Engineering 


Scm)ay,  October  23,  2005 

5-00  PM'7:00  PM  Registration  for  Tutorials  and  General  Conference 

(Tutorials  are  an  additional  $200  registration  fee) 


Monday,  October  24,  2005 

7:00  AM -S  PM  Registration 

y  /{jy  Continental  Breakfast  for  Tuforial  Attendees  ONLY 

(Tutorials  are  an  additional  $200  registration  fee) 


8:00  AM  ~5  PM 
12  Nom  '  1  PM 
1:00  PM  '5  PM 
5:00  PM  '6  PM 


Tutorial  Tracks  (Please  refer  to  following  pages  for  Tutorials  Schedule) 
Buffett  Lunch 

Tutorial  Tracks  (Please  refer  to  following  pages  for  Tutorials  Schedule) 
Reception  in  Display  Area  (Open  to  All  Participants) 


Tuesday,  October  25, 2005 

7:00  AM  Registration  &  Continental  Breakfast 

8:15  AM  Introductions 

Mr.  Sam  Campagna,  Director,  Operations,  NDIA 

8:30  AM  Opening  Remarks 

Mr.  Bob  Rassa,  Director,  Systems  Supportability,  Raytheon; 

Chair,  Systems  Engineering  Division,  NDIA 

8:40  AM  '  9:30  AM  Keynote  Address 

Mr.  John  London,  Deputy  Assistant  Secretary  of  Defense  (Nil) 

(C3ISR  &  IT  Acquisition) 

9:30  AM  '  10  AM  Break  in  Display  Area 

10:00  AM  '  12  Mwtpienary  Session:  Revitalization  of  Systems  Engineering  Within  DoD 

Moderator: 

Mr.  Mark  Schaeffer,  Deputy  Director,  Defense  Systems,  and  Director, 

Systems  Engineering,  OUSD  (AT&L) 

Panelists: 

Mr.  Terry  J aggers.  Director,  SAF/AQR  (Science,  Technology  &  Engineering) 

Mr.  CarlSiel,  ASN  (RDA)CHENG 
Mr.  Doug  Wilfsie,  US  Army  (Invited) 

Mr.  Kelly  Miller,  NSA  (Invited) 


12  Hook  -  1:30  PM  Luncheon  Speaker 

Mr.  Greg  Shelfon,  Vice  President,  Engineering  Manufacturing  Technology 
&  Quality,  Raytheon 

1:30  PM  '  5  PM  Concurrent  Sessions  (Please  refer  to  following  pages  for  session  schedule) 


5:00  PM  '  6:30  PM  Reception  in  Display  Area 


MotPay,  October  24,  2005 


KetjiTbratlon  St  Cowtiiwntai  Breakfast 

8:00  AM 

9:45  AM 

10:15  AM  12  HOOK  1:00  PM 

2:45  PM 

3:15  PM 

5  PM-6 PM\ 

TRACK  1 
Tutorial 

Session  1A1 


TRACK  2 
Tutorial 

Settlor  1A2 


TRACKS 

Tutorial 

Settlor  IAS 


TRACK  4 
Tutorial 

Settlor  1A4 


TRACK  5 
Tutorial 

Settlor  IAS 


TRACK  6 
Tutorial 

Settlor  1AG 


TRACK  7 
Tutorial 

Settlor  1A7 


ss 


Tt 


TRACK  8 
Tutorial 

Settlor  1A8 


How  to  Define  System 
Engineering  Processes  That  are 
Short  and  Usable 


Mr.  Tim  Olson,  Quality 
Improvement  Consultants,  Inc. 

Integrating  Systems 
Engineering  with  Earned  Value 
Management 


Mr.  Paul  Solomon, 
Northrop  Grumman  Corp. 


Up-To-Date  Systems 
Requirements  Tutorial 


Mr.  Jeffrey  Grady , 

JOG  Systems  Engineering,  Inc. 


Exploring  the  System  Solution 
Space  using  Behavior  Analysis 
and  Simulation:  Applying  M&S  to 
System  Engineering 


Mr.  James  Long,  Vitech  Corp. 

Systems/Software/Hardware 
Quality  Assurance 


Mr.  Al  Florence, 
The  MITRE  Corp. 


Innovative  D 

(DFSS)  Approaches  to  Test  and 
Evaluation:  A  Hands-On 
Experience 


Dr.  Mark  Kiemele, 

Air  Academy  Associates 


Object  Oriented  Systems 
Engineering  Methodology 
(OOSEM) 


Dr.  Abraham  Meilich, 
Lockheed  Martin 


TBA 


TRACK  1  How  to  Define  System 

Engineering  Processes  That  are 


Tutorial 


Short  and  Usable  (Continued) 


Mr.  Tim  Olson,  Quality 

Settlor  1B1  improvement  Consultants,  Inc. 


TRACK  2  lnte9rating  Systems 

Engineering  with  Earned  Value 

Tutorial  Man°3ement  (Continued) 


Mr.  Paul  Solomon, 

Settlor  1B2  Northrop  Grumman  Corp. 


TRACK  3  Up-To-Date  Systems 
Requirements  Tutorial 

Tutorial  (Continued) 


Mr.  Jeffrey  Grady, 

Settlor  IBS  JOG  Systems  Engineering,  Inc. 


TRACK  4  ExP'or'n9  the  System  Solution 
'  Space  using  Behavior  Analysis 
77 nthmAviJ  anc'  Simulation:  Applying  M&S 

to  System  Engineering  (Continued) 


Settlor  IBS- 


Mr.  James  Long,  Vitech  Corp. 


TRACK  C  Systems/Software/Hardware  Qual¬ 
ity  Assurance 
Tutorial  (Continued) 


Mr.  Al  Florence , 
Settlor  IBS  The  MITRE  Corp. 


TR  AC kS  C  Innovative  Design  for  Six  Sigma 
1  ^  ^  b  (DFSS)  Approaches  to  Test  and 
T,. j-mA /i /  Evaluation:  A  Hands-On  Experi- 

Tutorud  ence  (Continued) 


Dr.  Mark  Kiemele , 

Setturr  1BG  Air  Academy  Associates 


TRACK  7  Object  Oriented  Systems 
Engineering  Methodology 
Tutorial  (OOSEM)  (Continued) 


Dr.  Abraham  Meilich, 
Settlor  1B7  Lockheed  Martin 


TRACK  8 
Tutorial 

Settlor  1B8 


Any  -/  Systems  Engineering  Planning  - 
TRACK  7  A  Tutoria| 

Tutorial 


Col  Warren  Anderson,  OUSD 
Settlor  1C1  (AT&L)  Defense  Systems 

TR  AC kC  ?  Using  a  Measurement  Framework 
l  KA  z  Successfully  Achieve  Measur- 

Tutorial  ableResults 

Mr.  Tim  Olson,  Quality 
Settlor  1C2  Improvement  Consultants 

'tv  Arty  2  Requirements  Development  and 
l  RACK.  3  Management 

Tutorial 


Mr.  Al  Florence, 
Settlor  ICS  The  MITRE  Corp. 


TRACK  4 
Tutorial 


Air  Force  Integrated  Collabora¬ 
tive  Environment  (AF-ICE)  -  An  Air 
Force  and  Industry  Partner 
overview  and  update 


Mr.  Rick  Peters, 

Settlor  ICS  Air  Force  Material  Command 


TR  AC kC  C  T^e  Return  on  Investment  from 
L  KAUK  o  software  Engineering  Best 

Tutorial  Practices:  An  Introduction 


Mr.  Thomas  McGibbon, 
Settlor  1C5  ITT  Industries 


'TR  AC kC  £  What  Makes  A  Simulation 
J  b  Credible?  Cost-Effective  VV&A  in 

Tutorial  ^terns  Engineering  Process 


Mr.  David  Hall,  SURVICE 
Settlor  ICG  Engineering  Company 


TRACK  7  Object  Oriented  Systems 
Engineering  Methodology 
Tutorial  (OOSEM)  (Continued) 


Dr.  Abraham  Meilich, 
Settlor  1C7  Lockheed  Martin 


7 -r  as-ls  q  Performability  (Performance  and 
1  s  Reliability)  Modeling 

Tutorial 


Dr.  Meng-Lai  Yin, 

Settlor  1C8  Raytheon 


TR  AC kS  1  Systems  Engineering  Planning  - 
1  1  a  Tutorial  (Continued) 

Tutorial 


Col  Warren  Anderson,  OUSD 
Settlor  1 VI  (AT&L)  Defense  Systems 


TR  AC kC  7  Using  a  Measurement  Framework 
L  KA  2  Successfully  Achieve 

Tutorial  MeasurQble  Results  (Continued) 


Mr.  Tim  Olson,  Quality 
Settlor  1V2  Improvement  Consultants 


'rv  Any  2  Requirements  Development  and 
L  RACK  5  Management 

Tutorial  (Continued) 


Mr.  Al  Florence, 
Settlor  1VS  The  MITRE  Corp. 


TRACK  4  Air  Force  Integrated  Collabora- 
'  tive  Environment  (AF-ICE)  -  An  Air 

Tutorial  Force  and  lndustrY  Partner 

overview  and  update  (Continued) 


Mr.  Rick  Peters, 

Settlor  1VS  Air  Force  Material  Command 

TR  AC kS  C  Return  on  Investment  from 

l  KAUK  P  software  Engineering  Best  Prac- 

Tutorial  tices:  An  Introduction 


Mr.  Thomas  McGibbon, 
Settlor  IDS  ITT  Industries 


TR  AC kC  £  What  Makes  A  Simulation  Cred- 
l  KAUK  fo  jb|e?  Cost_Effective  VV&A  in  the 

'TuJ-/*,*; *  /  Systems  Engineering  Process 
lUtOTUU  (Continued) 


Mr.  David  Hall,  SURVICE 
Settlor  1VG  Engineering  Company 


TRACK  7  Object  Oriented  Systems 
Engineering  Methodology 
Tutorial  (OOSEM)  (Continued) 


Dr.  Abraham  Meilich, 
Settlor  1V7  Lockheed  Martin 


TR  AC kC  8  Performability  (Performance  and 
l  KAUK  8  Re|jability)  Modeling 

Tutorial 


Dr.  Meng-Lai  Yin, 

Settlor  1V8  Raytheon 
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Tuesday,  October  25,  2005 


1:30  PM 

3:00  PM.\ 

3:30  PM 

TRACK  1 

System*  Engtice&rtitg 
Effedttimve** 

The  Return  of  Discipline 

Technical  Planning  for  Acquisition 

Programs:  An  OSD  Perspective 

TRACK  1 

System*  Engtiveertiig 
EffedbiucnAss 

Implementation  of  Policy 
Requiring  Systems  Engineering 
Plans  for  Air  Force  Programs 
-  Results  and  Implications 

Systems  Engineering  Revitaliza¬ 
tion  at  SPAWAR  Systems  Center 
Charleston 

Engineering  for  Software 
Assurance 

Session,  201 

Dr.  Yvette  Weber, 

HQ  AFMC,  USAF 

Col  Warren  Anderson, 

OUSD  (AT&L)  Defense  Systems 

Session  2D1 

Mr.  Kevin  Kemper, 

US  Air  Force 

Mr.  Michael  Kutch,  Jr., 

SPAWAR  Systems  Center 

Ms.  Kristen  Baldwin, 

OUSD(AT&L) 

TRACK  2 

System *  Engtiieertitg 
Effedttvme** 

Technology  Readiness  Assessments:  A  Key 
Aspect  of  the  Systems 

Engineering  Process 

Taxonomy  of  Operational  Risks 

TRACK  2 

System*  Engtiieertitg 
Effedttim iass 

A  Method  for  Reasoning  About 
an  Acquisition  Strategy 

WBS  Based  Risk  Assessment 

Session  202 

Dr.  Jay  Mandelbaum, 

Institute  for  Defense  Analyses 

Mr.  Brian  Gallagher, 

Software  Engineering  Institute 

Session.  2D 2 

Mr.  Joseph  Elm, 

Software  Engineering  Institute 

Mr.  Bruce  Heim, 

(DCMA)  Boeing  Long  Beach 

TRACK  3 

Test  St  Evaluation  tit 
System *  Engtiteertiij 

Applying  the  Systems  Engineering 
Approach  to  the  Test  and  Evaluation 
Process 

Intelligent  Data  Analysis  Options  to  Support 
Aircraft/Ship  Systems  Testing 

TRACK  3 

Test  St  Evaluation  tit 
System*  Engtiteertiig 

Interweaving  Test  and  Evalu¬ 
ation  throughout  the  Systems 
Engineering  Process 

Recent  Innovations  in  Design 
for  Six  Sigma  (DFSS)  Testing 
Approaches  to  Speed 
Technology  to  the 

Marketplace 

Flight  Testing  Airborne  Radar 
Systems  to  Improve  System 
Performance 

Session  2C3> 

Mr.  Raymond  Beach, 

NAVAIR 

Mr.  Dean  Carico, 

Naval  Air  Warfare  Center 

Session  2D 3 

Mr.  Joseph  Tribble, 

AVW  Technologies 

Dr.  Mark  Kiemele, 

Air  Academy  Associates 

Mr.  Mark  London, 

NAVAIR 

TRACK  4 

Net  Oniric  Operations 

Guiding  DoD’s  move  into  the 

Information  Age 

Challenges  in  Development  of  System  of 
Systems  (SoS)  Architectures  in  a  Net  Centric 
Environment 

l 

TRACK  4 

Net  Centric  Operation* 

Real-Time  Tactical  Services  for 
the  GIG 

Next  Generation  Enterprise 
Information  Management 
Appliances 

Session  204 

Mr.  Jack  Zavin,  ASD(NII)/DoD  CIO 

Dr.  Abraham  Meilich, 

Lockheed  Martin 

£- 

Session  2D4 

Mr.  John  Noble, 

JHU  Applied  Physics  Laboratory 

Mr.  Michael  Lindow, 

The  MITRE  Corp. 

TRACK 5 

Loyitiics 

Intro  to  Logistics  &  Supportability 

Condition  Based  Logistics 

TRACK  5 

LogiTUc* 

FRACAS  Implementation  using 
ITLog 

Creating  a  Logistics  Health 
Management  System 

Session  205 

Mr.  Jerry  Beck, 

OSD  Office  of  ADUSD(LAMR) 

Mr.  Ron  Wagner, 

CoBaLt  Technology 

* 

Session  2D 5 

Mr.  William  Jacobs, 

Raytheon 

Mr.  Gary  O'Neill, 

Georgia  Tech  Research  Inst. 

TRACK  6 

Integrated  Diagnostics 

Intro  to  Integrated  Diagnostics 

Diagnostic  Software  -  What  your  average 
developer  doesn’t  know 

TRACK  6 

Integrated  Diagnostic* 

Designing  for  Health;  A 
Methodology  for  Integrated 
Diagnostics/Prognostics 

COTS-Based  Solution  for 
Integrated  Test  and 

Diagnostics 

Session  20G 

Mr.  Dennis  Hecht, 

The  Boeing  Company 

Mr.  Theodore  Marz,  Carnegie  Mellon  Uni¬ 
versity  -  Software  Engineering 

Session  2D6 

Mr.  Larry  Butler, 

Raytheon 

Dr.  Ion  Neag, 

TYX  Corp. 

TRACK  7 

System*  Safety 

Session  207 

System  Safety  in  Systems  Engineering  DAU 
Continuous  Learning  Module  Overview 

Ms.  Amanda  Zarecky, 

Booz  Allen  Hamilton 

System  Safety  in  the  Systems 

Engineering  Process 

Dr.  Ray  Terry, 

SURVICE  Engineering  Company 

TRACK  7 

System.  Safety 

Session  2D7 

Revitalizing  System  Safety  as 
One  of  the  Key  Elements  to 
Revitalizing  Systems  Engineer¬ 
ing  in  Department  of  Defense 
Acquisition  Programs 

Col  Warren  Anderson, 

OUSD  (AT&L)  Defense  Systems 

Linking  System  Safety  to 

Systems  Engineering 

Ms.  Paige  Ripani, 

Booz  Allen  Hamilton 

Integrating  MIL-STD-882 

Mr.  Rick  Milnarik, 

TRACK  8 

Software 

SuyportaJnUty 

Proper  Specification  of  Software  Require¬ 
ments 

C-17  Software  Development  Process 

TRACK  8 

Software 

SuyportaJnUty 

Successful  Verification  and 
Validation  Based  on  the  CMMI 
Model 

Automated  Software  Testing  Software  Supportability: 

Increases  Test  Quality  and  A  Software  Engineering 

Coverage  Resulting  in  Improved  Perspective 

Software  Reliability 

Session  208 

Mr.  Al  Florence, 

The  MITRE  Corporation 

Mr.  Hafez  Lorseyedi, 

The  Boeing  Company 

Session  2D 8 

Mr.  Tim  Olson,  Quality 
Improvement  Consultants,  Inc. 

Mr.  Frank  Salvatore,  High 
Performance  Technologies,  Inc. 

Mrs.  Stephany  Bellomo, 

SAIC 

Keceptim  tit  Display  Area, 
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KeptiiratioK  &  Ccnttitierytal  BreoAfatt 


8:1  SAM 

TRACK  1 

Systems  EngiKeerigg 
EjfeAtUmtess 

Tailorable  Decision  Analysis  and  Resolution 
process  and  tools  for  enterprise  wide 
application 

Defining  System  Development  Lifecycles 
to  Plan  and 

Manage  Projects  Effectively 

Session  3A1 

Mr.  Robert  Trifiletti,  Jr., 

US  Army  ARDEC 

Mr.  Bruce  Boyd, 

The  Boeing  Company 

TRACK  2 

Systems  Engitictirigg 
EjfeTbumiess 

Application  of  Risk 

Management  across 

Engineering  and  Acquisition 

Requirements  Engineering  Tips  and  Tricks 

Session  3A2 

Ms.  Rebecca  Cowen-Hirsch , 

Defense  Systems  Agency 

Mr.  Frank  Salvatore, 

High  Performance  Technologies,  Inc. 

TRACKS 

Systems  Engititierigg 
Effectiveness 

Effective  SE  Metrics  Tailored  to  the  Acquisi¬ 
tion  Life  Cycle 

Innovative  Procurement  Strategies 

Session  3  A3 

Ms.  Laura  Troiola, 

US  Army  -  ARDEC 

Mr.  David  Eiband, 

Defense  Acquisition  University 

TRACK  4 

Net  Centric  Operations 

Session  3 At 

Joint  Battle  Management  Command  & 
Control  RoadMap  -  Panel 

Moderators: 

Dr.  Vitalij  Garber,  Ms.  Robin  Quinlan,  DUSD 
(AT&L)  DS/SI 

Panelists: 

Maj  Gen  Charles  Simpson,  USAF 

MG  Michael  Vane,  USA 

Joint  Battle  Management  Command  & 

Control  RoadMap  -  Panel 

Moderators: 

Dr.  Vitalij  Garber,  Ms.  Robin  Quinlan,  DUSD 
(AT&L)  DS/SI 

Panelists: 

Maj  Gen  Charles  Simpson,  USAF 

MG  Michael  Vane,  USA 

TRACK  5 

LogitUc* 

Improving  Supportability  on  Currently 
Deployed  Weapon  Systems 

Process  for  Evaluating  Logistics  Readiness 

Levels  (LRLs)  for  Acquisition  Systems 

Session  3  AS 

Mr.  John  Sells, 

Tobyhanna  Army  Depot 

Mr.  Robert  Ernst, 

NAVAIR 

TRACK  G 

Modeling  St 
Susudation 

Improving  M&S  Support  to  Acquisition 

Improving  M&S  Support  to  Acquisition 
(Continued) 

Session  3 AG 

Mr.  James  Hollenbach, 

Simulation  Strategies,  Inc. 

Mr.  James  Hollenbach, 

Simulation  Strategies,  Inc. 

TRACK  7 

System  Safely 

A  Model  Linking  Safety,  Threat  and  Other 
Critical  Causal  Factors  to  Their  Mitigators” 
Relative  to  (Software,  Hardware,  and  Hu¬ 
man  System  Integration 

Mission  Sustainment  Through 

Acquisition  Environment,  Safety,  and 
Occupational  Health  (ESOH)  Risk 

Management 

Session  3A7 

Ms.  Janet  Gill, 

NAVAIR 

Ms.  Karen  Gill, 

Booz  Allen  Hamilton 

TRACK  8 

Software 

SuyportaTiUty 

Sustaining  Software-Intensive  Systems  -  A 
Conundrum 

Algorithm  Description 

Documentation  and  Validation  Process 

Session  3 AS 

Ms.  Mary  Ann  Lapham, 

SEI 

Mr.  Michael  K.  Bailey, 

Raytheon 

1 

10:15  AM 

TRACK  1 

System  Engineering,  Program  Manage- 

Tailoring  USAF  Systems 

ment  conjoined  Disciplines  over  the  Project  Engineering  for  the  Life  Cycle:  One  Shape, 

Systems  EnyUteeriny 

Life  Cycle 

Multiple 

Dimensions 

Effb&Urniett 

Session  3B1 

Mr.  William  Lyders, 

Mr.  Jeff  Loren, 

ASSETT,  Inc. 

MTC  Technologies,  Inc.  (SAF/AQRE) 

TRACK Z 

Engineering  and  Implementing  RMS  Engi¬ 
neering  DTC  Metrics 

System  Engineering  Metrics 

System *  Engdieerigg 

EffeAbumiett 

Session  3B2 

Mr.  Edward  Casey, 

Mr.  James  Miller, 

Raytheon  Missile  Systems 

United  States  Air  Force 

TRACK  3 

Using  Systems  Engineering 

Next  Generation  Combat  Systems  -  An 

Principles  to  Transform  R  &  D  Into  a  Military 

Overview  of  Key  Development  Concepts 

System *  Englntierigg 

System  Solution 

EffeAtumioss 

Session  3B3 

Dr.  James  Dill, 

Mr.  Matthew  Montoya, 

Foster-Miller 

The  JHU  Applied  Physics  Laboratory 

TRACK  4 

Network-Centric  Capabilities 

Testing  Net-Centric  Systems  of 

Development  for  Ground  Mobile  Forces 

Systems:  Applying  Lessons  Learned  from 

Net  Centric  Operation* 

Distributed  Simulation 

Session  3B4 

Ms.  Diane  Hanf, 

Mr.  R.  Douglas  Flournoy, 

The  MITRE  Corp. 

TRACK  5 

The  Management  of  Logistics  in  Large 

System  of  Systems  Analysis  of  Future 

Scale  Inventory  Systems  to  Support 

Combat  System  Sustainment  Requirements 

LogiTUos 

Weapon  System  Maintenance 

Session  3B5 

Mr.  Eugene  Beardslee, 

Mr.  Ivan  Wolnek, 

SAIC 

The  Boeing  Company 

TRACK  6 

Next  Generation  Manufacturing  Tech¬ 
nology  Initiative  and  the  Model-Based 

Problem  Space  Modeling 

ModeUnj  & 

Enterprise 

Simulation 

Session  3BG 

Mr.  Richard  Neal, 

Mr.  Jeffrey  O.  Grady, 

IMTI 

JOG  Systems  Engineering,  Inc. 

TRACK  7 

Army  Acquisition  Programs’ 

Current  DoD  Acquisition  Policies  and 

Installations,  Environmental,  Safety,  and 

Guidance  on  the  use  of  MIL-STD-882D  to 

System  Safety 

Occupational  Health 

Integrate  Environment,  Safety,  and  Occu¬ 

Considerations 

pational  Health  (ESOH)  Considerations  into 
the  Systems  Engineering  Process 

Mr.  Sherman  Forbes, 

Session  3B7 

Mr.  Donald  Artis,  Jr.,  Office  of  the 

USAF  -  SAF/AQRE 

DAS  A  (ESOH) 

TRACK  8 

The  Integration  of  Systems  Engineering  and 

The  Integration  of  Systems  Engineering  and 

Enterprise  Architecture  with  respect  to  the 

Enterprise  Architecture  with  respect  to  the 

Legacy  Systems 

Modernization  of  Legacy  Systems  -  Panel 

Modernization  of  Legacy  Systems  -  Panel 
(Continued) 

SwstauuMtint 

Session  3B8 

Mr.  Owen  Williams,  Science 

Mr.  Owen  Williams,  Science 

Applications  International  Corp. 

Applications  International  Corp. 

va 
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WexAtiesoLay,  October  26,  2005 


1:30  PM 

TRACK  1 

Systems  Engineering 
Effectiveness 

Architecture  Based  Systems 

Engineering  And  Integration 

A  Complementary  Approach  to  Enterprise 
Systems  Engineering 

Session  3C1 

Dr.  Rick  Habayeb,  Virginia  Tech 

Dr.  Brian  White ,  The  MITRE  Corp. 

TRACK  2 

Technical  Performance  Measures 

Turbo  Tax  for  Systems  Engineering 

Systems  Engineering 
Effectiveness 

Session  3C2 

Mr.  Jim  Oakes, 

BAE  Systems 

Mr.  Michael  Kutch,  Jr., 

SPAWAR 

TRACK  3 

Systems  Engineering 
Effectiveness 

Converting  High-Level  Systems 

Engineering  Policy  to  a  Workable  Program 

Revitalization  of  Systems  Engineering;  Past, 
Present  and  Future 

Session  3C3 

Mr.  James  Miller, 

US  Air  Force 

Ms.  Karen  Bausman, 

USAF  Center  for  Systems  Engineering 

TP  AC K  4  What  is  the  difference  between 

'  Multi-Level  Security  (MLS)  and  Multiple 

Net  Centric  Operations  Secure  Levels  (MSL)  Architectures  and  why 
r  do  you  care? 

A  Network  Centric  Warfare  Platform  With 

Multiple  Missions  in  Mind 

Session  3C4 

Mr.  Paul  Vazquez,  Jr., 

Raytheon  NCS 

Mr.  Peder  Jungck, 

CloudShield  Technologies 

TRACK  5 

LogitUos 

Reaping  the  benefits  of  PBL/CSL 

Priming  &  Tuning  the  ERP/MRO 

Engine:  Integrated  Through-life 

Supportability  Data  Management 

Session  3C5 

Ms.  Denise  Duncan, 

LMI 

Mr.  Patrick  Read, 

Pennant  Canada,  Ltd 

TRACK  6 

Update  on  SysML 

Data  Management  to  support  M&S 

Modeling  St 
Sinudation 

Session  3C6 

Mr.  Rick  Steiner, 

Raytheon 

Ms.  Denise  Duncan, 

LMI 

TRACK  7 

System  Safety 

Lessons  Learned  with  the  Application  of 

MIL-STD-882D  Within  the  Navy’s  Weapon 
System  Explosives  Safety  Review  Board 

Industry  perspectives  and  identified  barriers 

to  the  use  of  MIL-STD-882D  for  integrating 

ESOH  considerations  into  Systems 

Session  3C7 

Ms.  Mary  Caro, 

Naval  Ordnance  Safety  &  Security  Activity 

Mr.  Jon  Derickson, 

United  Defense 

TRACK  8 

Legacy  System* 

Sustainment 

The  Aging  Transport  Systems 

Rulemaking  Advisory  Committee:  Back¬ 
ground,  Results  and  Future  Impact  on  the 
Aviation  Industry 

Jammer  Integration  Roadmap 

Session  3C8 

Mr.  Kent  Hollinger, 

The  MITRE  Corp. 

Mr.  Adam  McCorkle, 

Georgia  Tech  Research  Institute 

330  PM 


§ 

§ 


TRACK  1 

Systems  Engineering 
EffeCtivenes s 

Implementing  SE  Processes  to 
Balance  Cost  and  Technical 
Performance 

A  Revolutionary  Model  to  Sup¬ 
port  Early  CAIV  Trades  and  Cost 
Predictions 

Session  3V1 

Dr.  Mary  Anne  Herndon,  SAIC 

Mr.  Bryan  Piggott,  InfoEdge 

TRACK  2 

Systems  Engineering 
Effectiveness 

A  Practical  Application  of  the 
Non-Advocate  Review 

Systems  Engineering  and  the 
Software  Laws  of 
Thermodynamics 

Unmanned  Aerial  Vehicle 
Survivability  Influence  on 

System  Life  Cycle  Cost 

Session  3V2 

Mr.  Bruce  Nishime, 

The  Boeing  Company 

Dr.  Thomas  Christian,  Jr., 

402  SMXG 

Mr.  Charles  Pedriani, 

SURVICE  Engineering 

TRACK  3 

Systems  Engineering 
Effectiveness 

AFRL  Systems  Engineering  System  Engineered  Research 

Initiative  -  Risk  Management  for  and  Development 

Science  and  Technology  Management 

Session  3V3 

Mr.  William  Nolte, 

USAF-AFRL 

Dr.  Steven  Ligon, 

SAIC 

TRACK  4 

Net  Centric  Operation s 

Systems  Engineering  Analysis 

and  Control  Methods  to  Assure 
Electromagnetic  Spectrum 
Access 

A  Strategy  for  Managing  the 

Development  and  Certification 
of  Net-Centric  Services  within 
the  Global  Information  Grid 

Session  3V4 

Mrs.  Renae  Carter, 

DISA  Defense  Spectrum  Office 

Mr.  Bernal  Allen, 

Defense  Systems  Agency 

TRACK  5 

Best  Practices  St 
Stator  delation 

On  the  Shoulders  of  CMM: 

CMMI  +  COTS  +  OA  +  nNIH  =  less 
(cost)  +  more  (capability) 

CMMI  for  Services 

Session  3  V5 

Mr.  Luke  Campbell, 

NAVAIR 

Mr.  Juan  Ceva, 

Raytheon  RIS 

TRACK  6 

Modeling  St 
Simulation 

Enterprise  Digital  Data 
Management 

The  Use  of  Simulation  in  the 
Management  of  Logistics  in 
Large  Scale  Inventory  Systems 
to  Support  Weapon  System 
Maintenance 

Ensuring  Accomplishment  of 
Performance  Based  Logistics 
Objectives  Using 

Model-Based  Systems  Engineer¬ 
ing 

Session  3V6 

Ms.  Cynthia  Hauer,  Millennium 
Data  Management,  Inc. 

Mr.  Eugene  Beardslee, 

SAIC 

Mr.  Timothy  Tritsch, 

Vitech  Corp. 

TRACK  7 

System  Safety 

Comparisons  and  Contrasts 

Between  ISO  14001,  OHSAS 
18001,  and  MIL-STD-882D  and 
their  Suitability  for  the  Systems 
Engineering  Process 

Evolution  of  Military  Standard 

882E 

USMC  Expeditionary  Fight- 

ing  Vehicle  (EFV):  A  Vehicle 
Designed  with  Environmental, 
System  Safety,  and  Occupa¬ 
tional  Health  (ESOH)  in  Mind 

Session  3V7 

Mr.  Kenneth  Dormer,  USAF 
Contractor  (SAF/AQRE) 

Mr.  Jimmy  Turner, 

Raytheon 

Ms.  Sandra  Fenwick, 

USMC  DR  PM  AAA 

TRACK  8 

Legacy  Systems/ 

Open  Systems 

NAVAIR  Integrated  In-Service 
Reliability  Program  -  Aging  Air¬ 
craft/Keeping  Legacy  Systems 
Viable 

Delivering  Effective  Solutions 
in  the  Age  of  Open  Source 
Technology 

Session  3V8 

Ms.  Debbie  Vergos, 

Naval  Air  Systems  Command 

Mr.  Edward  Beck, 

Computer  Sciences  Corp. 

5:30  PM 
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TRACK  1 

Systems  Enguteerigg 
Effe&Umvtst 

A  Systems  Affordability  Approach 

Using  Raytheon  Six  Sigma  Design 

Requirements  Engineering  Tips  and  Tricks 

Session  4A1 

Ms.  Yvette  Thornton, 

Raytheon 

Mr.  Frank  Salvatore, 

HPTI 

TRACK  2 

Systems  EnyUieeriny 
EjfedbUmiess 

Surveying  SE  Effectiveness 

Integrated  Survivability  Assessment  (ISA)  in 
the  Systems  Engineering  Process 

Session  4A2 

Mr.  Joseph  Elm, 

Software  Engineering  Institute 

Mr.  David  H.  Hall, 

SURVICE  Engineering  Company 

TRACK  3 

Systems  Engineering 
Effectiveness 

10  Golden  Questions  for  Concept  Explora¬ 
tion  and  Development 

The  C-17  Systems  Engineering 

Experience 

Session  4. A3 

Dr.  Dan  Surber, 

Raytheon  Technical  Services  Co. 

Mr.  Kenneth  Sanger, 

The  Boeing  Company 

TRACK  4 

Net  Centric  Operations 

Net  Centric  Test  &  Evaluation 

Profiling  and  Testing  Procedures  for  a  Net- 
Centric  Data  Provider 

Session  4A4 

Mr.  Ric  Harrison, 

DISA 

Mr.  Derik  Pack,  Space  &  Naval  Warfare 

Systems  Center  -  Charleston 

TRACK  5 

Best  Er&ctices  St 

Star} ordination 

Process  Architecture  and  Criteria  for  Les¬ 
sons  Learned 

Successful  Strategies  To  Improve  Your 
Requirements 

Session  4A5 

Mr.  Thomas  Cowles, 

Raytheon  Space  &  Airborne  Systems 

Mr.  Tim  Olson, 

Quality  Improvement  Consultants,  Inc. 

TRACK  6 

Modeling  St 
Simulation 

Application  of  a  State-Machine  Model  for 
the  Analysis  &  Optimization  of  Task-Post- 
Process-Use  [TPPU]  and  Task,  Process, 
Exploitation  and  Disseminate  [TPED] 
Processes 

A  Heuristics  Systems  Engineering  Approach 
to  Modeling  and  Analysis  of  the  U.S.  Strate¬ 
gic  Highway  Network  (STRAHNET) 

Session  4AG 

Mr.  Richard  Sorensen, 

Vitech  Corp. 

Mr.  Gerard  Ibarra, 

Southern  Methodist  University 

TRACK  7 

Education  St  Training 

Educating  Future  Systems  Engineers:  US  Mili¬ 
tary  Academy  Reception-Day  Simulation 
and  Optimization 

in  SE 

Session  4A7 

LTC  Simon  Goerger, 

Department  of  Systems  Engineering 

TRACK  8 

Net  Centric  Operations 

The  Role  of  the  Operator  and  System 
Engineer  in  the  Force  Modernization 
Environment 

TBA 

Session  4A8 

Mr.  Thomas  Nelson, 

Jacobs  Sverdrup 

Si  CcmMjj&wtal  Brenkfa^t 
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TRACK  1 

System*  Engineering 
EffeAtivme** 

How  the  Pro-Active  Program  (Project) 
Manager  uses  a  Systems  Engineer’s  Trade 
Study  as  a  Management  Tool,  and  not  just 
a  Decision-Making  Process 

Experience  in  Supporting  Systems  Engineer¬ 
ing  Project  Management  Using  CORE 

Session  4B1 

Mr.  Art  Felix, 

US  Navy 

Mr.  George  Blaine, 

United  Dfense,  LP 

TRACK  2 

System s  Engineering 
Effectiveness 

A  systems  approach  to  Accelerating  Test¬ 
ing,  a  case  study 

Applying  the  Systems  Engineering  Method 
to  the  Joint  Capabilities 

Integration  and  Development  System 
(JCIDS) 

Session  4B2 

Mr.  Douglas  Chojecki, 

Stewart  &  Stevenson,  TVSLP 

Mr.  Christopher  Ryder, 

JHU  Applied  Physics  Laboratory 

TRACK  3 

Systems  Engineering 
Effectiveness 

Performance-Based  System 

Architecture  Design  in  Global  Hawk  UAV 

X-47,  Joint  Unmanned  Air  Systems  (J-UCAS) 

Program  Update 

Session  4B3 

Mr.  Deepak  Shankar, 

Mirabilis  Design,  Inc. 

Mr.  Rick  Ludwig, 

Northrop  Grumman  Corp. 

TP  AC K  4  Joint  lnte9ratecl  BMC4I  Systems  Research 

for  Upgrading  Current  and  Legacy  BMC4I 
Net  Centric  Operation *  Systems 

Model  Driven  Architecture  -  Lessons 

Learned  in  Model  Assessments  for  Large 

Scale  Joint  Implementation 

Session  4B4 

Mr.  Billy  Bradley,  Jr., 

Raytheon  Integrated  Defense  Systems 

Ms.  Denise  Bagnall, 

Naval  Surface  Warfare  Center 

TRACK  5 

Best  Practices  St 

Start) ar delation 

Mature  and  Secure:  Creating  a  CMMI  and 
ISO/IEC  21827  Compliant  Process  Improve¬ 
ment  Program 

Performance-Based  Earned  Value 

Session  4B5 

Mr.  Michele  Moss, 

Booz  Allen  Hamilton 

Mr.  Paul  Solomon, 

Northrop  Grumman  Corp. 

TRACK  G 

Modeling  St 
Simulation 

Systems  Engineering  Approach  to 
Research,  Analyze,  Model  and  Simulate 
the  Interdependencies  of  Container 
Shipping  and  the  United  States  Critical 
Infrastructure  System-of-Systems 

Using  Commercial  Simulation  Software  to 

Model  Linear  and  Non-Linear  Processes:  US 

Military  Academy  Reception-Day 

Simulation  and  Optimization 

Session  4B6 

Ms.  Susan  Vandiver, 

Southern  Methodist  University 

LTC  Simon  Goerger, 

Department  of  Systems  Engineering 

TP  A CtC  7  Systems  Engineering  Professional  Develop¬ 
er  /  menf  Qnd  Certifjcation 

Education  St  Training 

in  SE 

Education  and  Training  in  Systems  Engi¬ 
neering  Support  Processes 

Session  4B7 

Mr.  Gerard  Fisher, 

The  Aerospace  Corp. 

Ms.  Cynthia  Hauer, 

Millennium  Data  Management,  Inc. 

TRACK  8  JCIP:  The  JBMC2  Roadmap’s 

SoSE-Based  Process  for 

Net  Centric  Operation s  Identifying  and  Developing 
r  Capabilities  Improvements 

Matrix  Mapping  Tool  (MMT) 

Session  4B8 

Dr.  John  Hollywood, 

RAND  Corp. 

Dr.  Judith  Dahmann, 

The  MITRE  Corp. 
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TRACK  1 
System*  Engineering 
Effectiveness 


_ 1:00  PM _ 

Standard  Approach  to  Trade  Studies  for  Effective  Implementation  of  Systems 
the  Systems  Engineer  Engineering  at  the  Aeronautical  Systems 

Center:  A  Systems  Engineering  Tool  Set 


Session  401 


TRACK  2 
System s  Engineering 
Effectiveness 


Mr.  Art  Felix, 
US  Navy 


Systems  Engineering  to  Enable 
Capabilities-based  Acquisition 


Mr.  Edward  Kunay, 
US  Air  Force 


Are  New  Acquisition  Programs  Taking  Lon¬ 
ger  to  Develop/Field  and  If  so  Why? 


Session  402 
TRACK  3 


Systems  Engineering 
Effectiveness 


Ms.  Kristen  Baldwin, 

OUSD/(AT&L)  DS/Systems  Engineering 

A  Systems  Architectural  Model  for  Man- 
Packable  Intelligence,  Surveillance,  and 
Reconnaissance  Micro  Aerial  Vehicles 


Dr.  Dennis  Strouble, 

Air  Force  Institute  of  Technology 

EW  Integration  Roadmap 


Maj  Joerg  Walter, 

Session  403  AFIT/SYE 


T'k  AC K  4  Enabling  Net  Centric  Capability  through 

Secured  Integrated  Networks  of  Modular 
Net  Centric  Operation*  and  Open  Architectures 


Mr.  Byron  Coker,  Jr., 
Georgia  Tech/GTRI 


Open  Systems  Architecture  &  Standard 
Interfaces  as  Mission  Capability  Enablers 


Session  404- 


TRACK  5 
Best  Br&ctices  St 
StaM^ at  dotation 


Dr.  Cyrus  Azani, 
OSJTF/NGC 


TBA 


Session  405 

TRACK  G 
Modeling  St 
Simulation 


MS2  Moorestown  Modeling  and 
Simulation  (M&S)  Support  Approach 


Mr.  William  Mish,  Jr., 
AMSEC 


What  CMMI  Can  Learn  From  the  PMBOK 


Mr.  Wayne  Sherer, 
US  Army  ARDEC 


Science-Based  Modeling  and  Simulation 
on  DoD  High  Performance  Computers 


Session  400 
TRACK  7 


Mr.  David  Henry, 
Lockheed  Martin  MS2 


Training  Your  Systems  Engineering  Work¬ 
force 


Education  St  Training 


in  SE 


Dr.  Larry  Davis,  High  Performance 
Computing  Modernization  Program 

Filling  the  Expertise  “Gap” 


Mr.  Michael  Kutch,  Jr.,  Mr.  John  White, 

Session  407  SPAWAR  US  Air  Force 

TRACK  8  TBA  TBA 

Net  Centric  Operation* 


Session  408 


STRENGTH  THROUGH  INDUSTRY  &  TECHNOLOGY 


2111  Wilson  Blvd. 

Suite  400 

Arlington,  VA  22201-3061 
www.ndia.org 

Promotional  Partner: 


<B> 


An  advanced  weapon  and  space  systems  company  with  sales  of  ap¬ 
proximately  $3B  and  strong  positions  in  propulsion,  composite  structures, 
munitions  precision  capabilities,  and  civil  and  sporting  ammunition.  The 
company  is  the  world’s  leading  supplier  of  solid  rocket  motors  and  the 
nation’s  largest  manufacturer  of  ammunition.  ATK  is  a  $3.1  billion  ad¬ 
vanced  weapon  and  space  systems  company  employing  approximately 
14,500  people  in  23  states. 

Building  Proven  Reliability:  ATK  rocket  motors  represent  a  national  asset, 
offering  an  affordable  and  sustainable  way  to  implement  America's  new 
space  exploration  initiative. 

Reaching  New  Frontiers:  AK  space  systems  are  vital  to  reaching  new  fron¬ 
tiers  in  space  and  furthering  our  knowledge  of  the  universe. 

Providing  Homeland  Security:  ATK  advanced  technologies  and  law  en¬ 
forcement  ammunition  are  critical  to  America’s  efforts  to  defend  our 
homeland  and  our  citizens. 

Expanding  Platform  Capabilities:  ATK  advanced  weapon  systems  are 
expanding  the  capabilities  of  today’s  ships,  aircrafts,  and  ground  vehicles 
-  and  are  preparing  the  way  for  the  platforms  of  tomorrow  and  beyond. 
Defending  our  Nation:  ATK  ammunition  for  the  U.S.  armed  forces  is  play¬ 
ing  a  key  role  in  the  global  war  on  terrorism. 

Find  out  more  at  www.atk.com. 


8th  Annual  Systems  Engineering  Conference  &  Exhibition 
“Focusing  on  Mission  Areas,  Net-Centric  Operations  and  Supportability  of  Defense  Systems” 

October  24  -  27,  2005 
San  Diego,  CA 


Evolution  of  MIL-STD-882E 


Bob  McAllister,  USAF 
Jimmy  Turner,  Raytheon 


•  Long  ago 

-  Analyses  done  after  the  fact 

•  Ballistics  Sys  Div  Exhibit  62-41  (1962) 

-  Ballistic  missiles 

•  MIL-S-38130A  (June  1966  and  March  1967) 

-  Aircraft,  space,  &  electronics 

•  MIL-STD-882  (July  1969) 

-  Mgmt  emphasis  &  industry  involvement 

•  MIL-STD-882A  (June  1977) 

-  Hazard  probabilities  and  risk  acceptance 

•  MIL-STD-882B  (Mar  1984  and  July  1987) 

-  Individual  tasks 

•  MIL-STD-882C  (Jan  1993  and  Jan  1996) 

-  Integrated  hardware  and  software  tasks 

•  MIL-STD-882D  (Feb  2000) 

-  Acquisition  reform 


Risk  Levels  &  Matrices 


•  MM-S-38130A 

-  No  levels  nor  matrix 

•  MIL-STD-882 

-  No  matrix.  Defined  hazard  levels 

•  MIL-STD-882A 

-  No  matrix  -  reversed  hazard  levels. 

-  New  qualitative  probability  levels 

•  MIL-STD-882B 

-  Qualitative  risk  matrices  in  appendix 

•  MIL-STD-882C 

-  Qualitative  and  quantitative  matrices  in  Appendix. 

-  Established  risk  acceptance  levels 

•  MIL-STD-882D 

-  Qualitative  matrix,  but  quantitative  probability  levels. 

•  MIL-STD-882E  (draft) 

-  Multiple  matrices  and  risk  levels 


Qualitative  matrix  (-882B) 


FREQUENCY  OF 

OCCURRENCE 

HAZARD  CATEGORIES 

I 

CATASTROPHIC 

II 

CRITICAL 

III 

MARGINAL 

V 

NEGLIGIBLE 

(A)  FREQUENT 

1A 

2A 

3A 

4A 

(B)  PROBABLE 

IB 

2B 

3B 

4B 

(C)  OCCASIONAL 

1C 

2C 

3C 

4C 

(D)  REMOTE 

ID 

2D 

3D 

4D 

(E)  IMPROBABLE 

IE 

2E 

3E 

4E 

iJp  Quantitative  Matrix  (-882C) 


HAZARD 

CATEGORY 

FREQUENCY 

(1) 

CATASTROPHIC 

(2 

CRITICAL 

(3) 

MARGINAL 

(4) 

NEGLIGIBLE 

(A)  FREQUENT 

(X>  10-1)* 

1A 

2A 

3A 

4A 

(B)  PROBABLE 

(10-1  >X>  10-2  )* 

IB 

2B 

3B 

4B 

(C)  OCCASIONAL 
(10-2  >X>  10-3  )* 

1C 

2C 

3C 

4C 

(D)  REMOTE 

(10-3  >X>  10-6  )* 

ID 

2D 

3D 

4D 

(E)  IMPROBABLE 

(10-6  >X)* 

IE 

2E 

3E 

4E 

* 


Example  of  quantitative  criteria 


Qualitative  Matrix  (-882D) 


TABLE  A-III.  Example  mishap  risk  assessment  values. 


SEVERITY 

Catastrophic 

Critical 

Marginal 

Negligible 

PROBABILITY 

Frequent 

1 

3 

7 

13 

Probable 

2 

5 

9 

16 

Occasional 

4 

6 

11 

18 

Remote 

8 

10 

14 

19 

Improbable 

12 

15 

17 

20 

Probability  Levels  (-882D) 


•  Frequent 

•  Probable 

•  Occasional 

•  Remote 

•  Improbable 


more  than  10'1 
between  10'2  and  10'1 
between  10'3  and  10'2 
between  10‘6and10'3 
less  than  10'6 


882D:  Numbers  are  for  individual  item,  not  fleet 
882C:  Doesn’t  specify 


Origin  of  numbers? 


•  Done  by  committee  (like  a  camel) 

•  Not  enough  probability  levels  to  change  single  order 
of  magnitude  (skipped  ahead  from  10'3  to  10'6) 

•  Why  1 0'6? 

-  Originated  in  munitions  world 

-  Seemed  ‘unapproachable.  (‘Not  one  in  a  million!’) 


•  MIL-STD-882D  complied  with  Acquisition  Reform 
-Tells  ‘what’  to  do,  not  ‘how’ 

-  Specifies  eight  generic  system  safety  steps 

=  Have  a  plan 
=  Identify  hazards 
=  Assess  their  risks 
=  Take  action  on  the  risks 
=  Accept  residual  risks 

-  882  D  removed  the  882C  System  Safety  Tasks 

-  Considered  to  be  too  ‘watered-down’ 

*  We  overdid  it,  so  need  a  more  robust  standard 


•  Mid  2004,  first  draft  MIL-STD-882E 

-  Re-instated  System  Safety  Tasks 

-  Re-instated  software  criticality  matrix 

-  Changed  Mishap  Risk  Assessment  Value  (MRAV)  to 

Mishap  Risk  Index  (MRI) 

•  Early  2005,  Second  draft 

-  Add  new  Tasks  on  Safety  Critical  Functions  and  FHAs,  etc 

-  Re-instate  Task  usage  matrices 

-  Re-instate  “F”  probability  level  (designed  out/impossible) 

-  Revised  the  risk  matrices 

=  $10K  to  $20K 
=  Expanded  ‘Low  risk  range’ 


Next? 


•  Summer  2005,  third  draft 

-  Re-structuring  for  better  logic  flow 

-  Multiple  risk  matrices  -  upper  right  is  High 

-  New  precedence  step  -  added  Engineering  Safety  Features 

(Examples  include  the  emergency  core  cooling 
system  of  a  nuclear  reactor  and  loss-of-tension  braking 
for  elevators;  full-time,  on-line  redundant  paths; 
interlocks;  ground-fault  circuit  interrupters  and 
uninterruptible  power  supplies) 

-  Five  system  safety  ‘Elements;  instead  of  8  Steps 

•  Being  coordinated  by  GEIA  G-48  (System  Safety)  Panel 

•  Publish,  fall/winter  2005 


National  Defense  Industrial 
Association 

8th  Annual  Systems  Engineering 
Conference 


Jammer  Integration  Roadmap 

(Unclassified) 
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GTRI 


Adam  McCorkle, 
adam.mccorkle@gtri.gatech.edu 
(404)  894-2508 


Georgia 

Tech 


•  F-16C+ 

•  ANG 

•  AFRC 

•  AO/A- 10 

•  ANG 

•  AFRC 

•  ACC 


Georgia 

Tech 
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Integrated  Platforms 
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Out  With  The  Old 


• 

The  C- 

9492  is 

a 

Replacement  f 

or  the 

C-6631 

L  Analog 

l  Control 

Head 

• 

The  C- 

9492  Controls 

the 

ECM  Pod  via  a 

28V 

Discrete  Powe 

r  Signc 

>1,  a 

Clock  ‘ 

signal,  and  a  F 

’ulse 

Position  Data  ( 

;ppd)  1 

line 

• 

PPD  is 

a  Serial 

,Bi- 

Directional ,  Time 
Multiplexed  Data  Bus 

Georgia 

Tech 
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•  SWV  1.0B3  was  the  First 
Fielded  Version  of  the  ALQ- 
213  Software  (1998) 

•  SWV  1.0B3  Supported  the 
PPD  Interface  for  the  ALQ- 
131,  ALQ-184,  &  ALQ- 
184(V)9  ECM  Pods 

•  SWV  2.0B5  is  Currently  in 
Flight  Test.  This  is  the 
Introduction  of  the  Threat 
Response  Processor  (TRP) 

•  SWV  3. OF  of  the  ALQ-213 
Begins  the  1553  Integration 
Between  the  Control  Head 
and  the  ECM  Pods  (2004) 


Georgia 

Tech 
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v  $ 


ALR-69  /  ALR-69A 


ALE-47 


COMET 


Georgia 

Tech 


DDTl®®t!Efla® 


MIL-STD-1553 
Other  (Discrete  Signals  &  Serial  Busses) 


ALQ-131 


TARS 


DVR 


AVIONICS 
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ALQ-213  SWV  3. OF 


Engineering  Release 

I  (3. 

0F1) 

• 

December  2004 

• 

Initial  Polling  of; 

ALQ- 

131  1 

En 

gineering  Release 

II  (3 

!.0F2) 

• 

May  2005 

• 

Initial  Polling  of; 

ALQ- 

184 

• 

Introduction  of  P 
Data 

d_Q-1 

L31  St 

Engineering  Release  III  (3.0F3) 

•  September  2005 

•  Introduction  of  ALQ-184  Status  Reporting  with  1553 
Data 

•  Refinements  made  to  ALQ-131  Status  Reporting 

•  Introduction  of  ALR-69/ALQ-131  Correlation 


Georgia 

Tech 


[RiSSOglReffi] 

DDT®aaMia® 


Georgia 

Tech 


DDTl®®t!Efla® 


•  Configuration 
•  Status 

•  Subband  Freq  Limits 

•  Pod  Instrumentation 
•  Jamming  Activity 


ALQ-213  Polling  of  Jammer  Data 


•Configuration 
•  Status 
•  Track  Files 
Preset  Jamming 


ALQ-131 


ALQ-184 


Jammer  Integration  Roadmap 


FY  ‘05  FY  ‘06  FY  ‘07  FY  ‘08 


Georgia 

Tech 
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Phase  I 


Development  and  Installation  of  the 
1553  Hardware  Kits 

Definition  of  Interfaces 

Formation  of  Integration  Working 
Group 

ALQ-213  Polling  of  Jammer  Data 
Compliance  with  Defined  1553  ICDs 


Phase  II 


ALQ-213  Control  of  Pod  Modes  with  1553 

Correlate  Threat  Identification  with  ALQ-213 

Update  Mission  Data  Tools  for  Correlation 

Increase  Pod  R&M  Data  for  Post  Mission 
Maintenance 

Coordinate  Jamming,  Dispensing,  and 
Aircraft  Maneuvers  for  Optimized  Responses 

Incorporate  Pod  Reprogramming  via  1553 

Provide  Jamming  Indication  on  ALR-69 
Display 


Georgia  J , 

Tech  m  DtrasOaMia® 
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Phase  III: 


•  Incorporate  Jammer  Threat  Identification  to 
Resolve  ALR-69  Ambiguities 

•  Remove  Jammer  Interference  From  RWR 
Display 

•  Optimize  Jamming  Response  via  ALQ-213 
TRP 

•  Remove  Jammer  Interference  From  Fire 
Control  Radar  (FCR)  Display 

•  Send  Data  to  RWR  for  Direction  Finding 
(DF) 

•  Incorporate  Real-Time  Pod  Status  for  ALQ- 
213  TRP  Compensation 


Georgia  J , 

Tech  DDT®aaMia® 
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Phase  IV:  1 


•  Incorporate  Advanced  Location  Systems 

•  Optimize  the  Integrated  EW  Suite  for  Threat 
identification  and  Warning 

•  Incorporate  Advanced  Chaff  and  Jamming 
Techniques 

•  Enable  Cooperative  Jamming  with  Multiple 
Jammers 

•  Incorporate  Real  Time  Pod  Status  for  Pilot 
Go/No-Go  and  Fault  Analysis 

•  Provide  Advanced  ECM  Techniques  Directed  by 


TRP 


Georgia  J , 

Tech  DDT®aaMia® 
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Threat  Identification  for  ALQ-213  Correlation  A 


i 


Threat  Identification  will  Lead  to  the  Following 
Benefits: 

•  Identification  of  Jammed  Threats  on  RWR 

•  Optimized  Threat  Response 

•  Resolution  of  RWR  Ambiguities 

•  Declutter  of  RWR  Display 


BEFORE  INTEGRATION 


AFTER  INTEGRATION 


Georgia  J , 

Tech  m  DtrasOaMia® 


Real-Time  Pod  Status  for  TRP  Compensation  A 


i 


1553  Jammer  Status  Messages  will  Allow 
TRP  to  Select  Jammer  Techniques  Based 
on  the  Specific  Health  of  the  Pod 

Current  Functionality  Only  Allows  the  TRP 
to  Base  Decisions  on  Jammer  Presets 
When  the  R/P  is  Non-Functional 

TRP  Logic  Must  Consider  "Age-In"  and 
"Nuisance"  Faults 

Refined  Decisions  can  be  Made  by  the  TRP 
on  a  Band,  Sub-Band,  or  Channef  Level 


Georgia  J , 

Tech  m  DtrasOaMia® 


I  Advanced  ECM  Techniques  A 

•  Combining  Jamming  and  Dispense 
Programs  to  Increase  Survivability 

•  Critical  Elements  include:  Timing, 

Order,  and  Resource  Management 
by  the  ALQ-213 

•  Time  Resolution  of  Combined 
Techniques  is  Critical  When 
Transferring  Between  Jamming  and 
Dispensing 

•  Examples:  Illuminated  Chaff,  Terrain 
Bounce 


Georgia 

Tech 
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LOCKHEED  MARTIN 


Challenges  in  Development  of  System  of  Systems 
(SoS)  Architectures  in  a  Net  Centric  Environment 

Abe  Meilich,  Ph.D.,  C.C.P. 

Lockheed  Martin  Integrated  Systems  and  Solutions 
Net  Centric  Integration,  System  of  Systems  Engineering) 


j  JDJA  on  C0n.f3.r3.rj 03  Oo'iobsr  20U5 


Agenda 

•  Challenges  of  Systems  of  Systems  (SoS) 
Engineering  -  Implications  on  Scope  and 
Management  of  the  Net  Centric,  DOD 
Enterprise 

•  How  to  use  DODAF  to  help  create  a  SOA 
architecture 

•  SoS  Interoperability 

•  Network  Centric  Operations  Industry 
Consortium  (NCOIC)  support  to  SoS 
architecture  standards 


NDIA  SE  Conference  October  2005 


Abe  Meilich,  Ph.D. 


Page  2 


Some  Observations  on  Architecting  SoS 

•  “SoS  [engineering]  may  not  turn  out  to  be  primarily  an 
engineering  field.” 

•  “Systems  engineering  is  based  on  the  assumption  that  if  given 
the  requirements  the  engineer  will  give  you  the  system.” 

Source:  “System  of  Systems  Symposium:  Report  on  a  Summer  Conversation”,  November  2004, 

Potomac  Institute  for  Policy  Studies. 

•  How  do  we  set  boundaries  in  order  to  create  a  defendable  set  of 
requirements? 

H  Allow  scope  expansion  but  build  a  flexible  interface  specification 
according  to  requirements  we  need  to  vision  today? 

-  What  is  context  of  data  behind  interface? 

•  Is  the  spiral  approach  low  risk  and  the  best  approach? 

Dependent  on  robust  mf  restructure  [e.g.,  GIG,  NCES,  NCOE,  etc.]  is  in 
place,  mission  applications  can  evolve  their  functionality 

Most  likely,  evolution  through  Darwinian  survival  will  be  the  long  term 
trend 
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Some  Observations  on  Architecting  SoS 

•  Static  designs  with  well  defined  specifications  worked  very 
goodin  a  stove-piped  environment 

■  Net  Centric,  flexible  solutions  can  no  longer  follow  this 
paradigm  and  expect  to  survive 

•  Optimality  and  efficiency  is  not  as  important  as  run-time 
interoperability  with  services  that  were  not  envisioned  at 
design  time  Inflexibility,  compose-ability,  extensibility  are 
now  much  more  important 

•  “■..processes  that  have  good  asymptotic  properties,  and 
that  can  evolve  to  keep  performing  in  unstable 

environments. are  the  properties  that  one  really 
desires  for  longevity  in  hostile,  asymmetric  environments 

•  Will  architecture  frameworks  like  DODA1  be  sufficient  to 
help  us  do  this? 

-  Growing  recognition  that  DODAF  (in  its  present  form)  is 
insufficient  to  capture  the  SoS  emergent  behavior  -  it 
probably  shouldn’t? 

•  The  dynamics  of  cognitive  and  social  processes  do  not 
obey  static  representations  and  rules  of  architecture 

it 

“System  of  Systems  Symposium:  Report  on  a  Summer  Conversation”,  November  2004,  Potomac  Institute 
for  Policy  Studies. 
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Some  Observations  on  Architecting  SoS 

•  It  has  been  noted  that  the  only  way  to  really  SE  a  SoS  is 
to  experiment  as  the  system  evolves  as  opposed  to 
“design”  the  system. 

-  “Rapid  experimentation  will  be  more  effec|ve  than 
attempting  to  create  a  master  plan  fola  complete 
solution.”1 

-  ...  by  asking  and  observing  what  people  do  and  providing 
them  with  evolving  prototypes,  the  architect  can  identify 
and  validate  what  people  find  useful  and  therefore 
provides  value  to  the  enterp  rase,”  1 

•  Traditionally,  single  systems  designed  for  specific 
context  and  specific  missions;  SoS  has  changing 
context  and  has  to  adapt  to  changing  missions 

-  Solution?  Leverage  Family  of  Systems  (FoS)  approach 

•  But  -  Can  we  afford  its  complexity? 

-  Less  expensive  to  spiral  software  than  spiral  physical 
systems 

-  Can  M&S  save  cost  and  will  it  be  affordable  for  complex 
systems? 

1  Goodhart,  Brian  and  McCabe,  Rich.  “What  Is  Enterprise  Architecture?”,  SPC,  2004 
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Some  observations  on  Architecting  SoS 

*  Systems  tend  to  be  architected  based  on  workflow 

-  Look  at  today’s  most  popular  enterprise  architecting  practices 
(i.e.,  engineer  human  processes  similarly  to  any  other  system 
component:  as  sequences  of  actions  with  measurable  inputs 
and  outputs  —  that  is,  a  workflow) 

•  The  precision  and  clarity  of  specification  possible  with 
this  approach  is  necessary  for  hardware  or  software, 
but,  as  [Pajerek  2000]  shows,  is  not  terribly  helpful  for 
human  only  processes  and  easily  becomes  a 
drawback. 

-  “Only  the  simpler,  more  straightforward  processes  lend 
themselves  to  a  workflow  treatment,  and  by  and  large,  these 
tasks  should  be  automated  entirely  to  free  up  people  to 
concentrate  on  the  creative  tasks  where  they  are  needed 
most.”1 


Pajerek,  Lori.  "Processes  and  Organizations  as  Systems:  When  the  Processors  are  People,  Not  Pentiums.' 
Systems  Engineering:  Journal  of  the  International  Council  on  Systems  Engineering  3:  (June  2000). 
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Some  observations  on  Architecting  SoS 

*  “...Most  SoS  problems  involve  open  systems  which  lack 
a  clear  boundary.  Our  existing  tool  set  mostly  reqffires 
closing  the  problem  by  defining  some  boundary  and 
assuming  no  surprises  come  from  the  outside...” 

*  “Better  tools  are  needed  by  the  SoS  community  .... 

While  emergence  has  been  a  source  of  fascination  for 
the  complexity  community  for  some  time,  we  still  do  not 
know  how  to  deal  with  emergent  phenomena  in  a 
rigorous  wav.” 

*  “A  third  challenge  area  is  that  of  dealing  with  systems 
that  include  autonomous  agents.  At  least  part  of  the 
reason  SoS  differs  from  classically  understood  systems 
engineering  is  that  all  SoS-tvpe  networks  necessarily 
contain  people  and  perhaps  other  types  of  agents.  The 
behavior  of  agents  cannot  be  dictated  by  the  engineer: 
agents  can  take  on  a  life  of  their  own,  so  to  speak.  This 
is  one  of  the  big  reasons  unexpected  phenomena  can 
emerge  in  SoS  situations.” 

Source:  “System  of  Systems  Symposium:  Report  on  a  Summer  Conversation”, 

November  2004,  Potomac  Institute  for  Policy  Studies. 
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Systems  Exist  at  Different  Levels 
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Competing  in  tie  Information-Age 

...the  Operations 


Soda 


NDIA  SE  Conference  October  2005 


Abe  Meilich,  Ph.D. 


Page  13 


Close  Air  Support  Mission 
Domain  Overlay 


Operator 


n 


Display 


Knowledge  Information 


Host 


Box-to-Box 


Terminal  Terminal 

(SADL) 


Host 


Data 


Data 


Information  Exchange 
SllShared  Awareness 


Legend: 


Technical 


Procedural 


Operator 


Display 


Information 


Knowledge 


Operational 


NDIA  SE  Conference  October  2005 


Abe  Meilich,  Ph.D. 


Hypotheses!  The  NCW  Value  Chain 


Information 

Domain 


Robustly 
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Force 
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Domain 


Quality  of 
Information 


Physical 

Domain 


Mission 
Effectiveness 


Information  Domain 
Cognitive  Domain 

Social  Domain 

Physical  Domain 


NDIA  SE  Conference  October  2005 


Abe  Meilich,  Ph.D. 


Page  15 


Implications  for  NCW  SoS  Systems 

Engineering 

•  SoS  Engineering  is  a  consolidated  discipline  that 
borrows  from: 

-  System  Engineering  (Physical  and  Information  Domain;  and 
Structured  management  of  other  disciplines) 

-  Operational  Analysis  (All  Domains) 

-  Decision  Analysis  (Physical,  Information,  and  Cognitive 
Domains) 

-  Modeling  and  Simulation  (All  Domains) 

-  Value  Engineering  (All  Domains) 

-  Cognitive  Modeling  (Cognitive  Domain) 

-  Collaboration  Theory  (Social  Domain) 


Implication:  Training,  competency,  and  domain  knowledge 
beyond  present  common  application  of  these  disciplines 
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Vision  for  the  Future 
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Determine  how  to  use  Service  Oriented  Architecture  (SOA)  concepts 
in  support  of  achieving  net-centricity  in  a  multi-service  environment 

Source:  “Developing  Architectures  in  a  Cross  Service  Environment”  ,  Murray  Daniels  (MITRE) ,  28  Sept  2004 
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Service  Oriented  Architecture  (SOA) 

Service-Oriented  Architecture  is  architectural 
style  whose  goalis  to  achieve  loose  coupling1 
among  iliteracti^g  services2 


New  set  of 
Problems 
here 


1  Loose  coupling  describes  the  configuration  in 

which  artificiafl  dependency  has  been  reduced  to 
a  minimum 

2  A  service  isa  set  of  actions  that  form  a  coherent 

whole  for  blkhservice  providers  and  service 
requesters 


NDIA  SE  Conference  October  2005 


Ph.D. 


Page  18 


Service  Oriented  Architecture 


Service  Produc 

Data  and  applications 
available  for  use,  accessible 
via  services.  Metadata  added 
to  services  based  on 
producer’s  form  at. 


•  Describes  content  using  metadata 

•  Posts  metadata  in  catalogs  for  discovery 

•  Exposes  data  and  applications  as  services 


In  voke 


Service  Consum  er 


Automated  search  of  data  services 
using  metadata.  Pulls  data  of 
interest.  Based  on  producer 
registered  format  and  definitions, 
translates  into  needed  structure. 


•  Searches  metadata  catalogs  to  find  data 


services 


•  Analyzes  metadata  search  results  found 

•  Pulls  selected  data  based  on  metadata 


understanding 


Publish 


Enabled  Ini 


S  e  rvice 
R  eg  istries 


Messaging  Data 

Services  Services 


D  is  cover 


T  ra  n  sfo  rm  atio  n 
Services 


services 


SOA  Arc 
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Operational 

Thread 

Sub 

Activities 


* 


Operational 


Interop 


Use  Case 


Composite 

Applications 


^££13 


Functions 


Sub 

Functions 


System 


Platforms  -*• 

Modified  from: 


“Developing  Architectures  in  a  Cross  Service  Environment”  ,  Murray  Daniels  (MITRE) ,  28  Sept  2004 
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Roadmap  Platform  Information  System  Activities 

Allocation  Services  Function 


Example 


Architecture  data 

•  Analyze  Unplanned 
Immediate  Targeting 
Opportunities 

-  Build  Watch  List 

•  Target  Information 
Management 

-  Target  List  Management 

•  Target  List  Management 
Service 

-  addT  argetsToList 
isTargetlnList 

•  Target  List  Service  allocated 
to  (e.g.  AOC,  JSTARS, 

-) 


Feeds 


Mapping 


Service  Arch 

Use  Case,  OV-5^6 
SV-5 
SV-4 


SV-10,  SV-6 


SV-1 


Roadmap 


Source:  “Developing  Architectures  in  a  Cross  Service  Environment”  ,  Murray  Daniels  (MITRE) ,  28  Sept  2004 
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AV-2 


Growing  Importance  of  Interoperability 

•  Network  Centric  warfighting  concepts  push 
systems  towards  greater  interaction  (and 
dependency!) 

•  Advent  of  the  GIG  ncreasingly  makes 
systems  accessible  to  one  another 

•  Grow  ng  experience  with  coal  tion 
operations  drives  coalition  interoperab  lity 

•  Commercial  adoption  of  the  Internet 
increases  customer  “sense  of  the  possible” 

NDIA  SE  Conference  October  2005  Abe  Meilich,  Ph.D.  Page  22 


DODAF  Views  and  Interoperability  Assessment  Criteria 


UJTLs 


Functional 
Capability  Levels 


Which  Systems 
interact? 

About  what? 
How  much? 
And  why? 

To  what  effect? 


Low 


Operational 

View 


Identifies  Participant  Relationships 
and  Information  Needs 

i _ i 

Battlespace 
Representation  and 
Naming  standards 


Technical 
Feasibility  Levels 

Can  capability  be 
achieved  with 
current  stds  & 
technologies? 
Are  new 
standards 
needed? 

Is  the  information 
obtainable, 
Accurate,  timely? 


Low 


Data  models,  process 
algorithms 


Systems 

View 

Relates  Capabilities/Characteristics 
to  Operational  Requirements 

Net-Ready 
Levels 


Data  element  standards, 
Protocols,  environments 


What  do  systems  say  to  each  other? 
How  is  this  information  represented? 
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Technology 
readiness  levels 


Low 
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Technical  Views 


Prescribes  Standards 
and  Conventions 

i _ i 

How  do  systems  interact? 

What  standards  are  used? 
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How  should  we  tackle  the  SOS  Si  future? 


•  Process 

-  Update  our  SoS  SE  processes  for  a  NC  environment  to  guide 
us  internally  (within  our  companies)  and  externally  (e.g.,  for 
DOD:  JCIDS  3170,  DODI  4630,  DOD  5000.2,  etc.) 

-  Share  ideas  presented  here  and  conduct  further  research  in 
SoS  SE,  SoS  Architecture  development  and  SoS/FoS 
utilization 

»  Business  Model  -  Openness  must  be  balanced  with  competition 
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How  should  we  (DOD  and  Contfactors)  tackle 

the  SOS  SE  future? 

•  Implementation 

-  Participate  in  evolving  Consortiums  (NCOIC, 

W2COG,  NCOIF,  etc.)  that  will  help  set  standards  for 
architecture  and  systems/services  development  on 
the  GIG,  for  example: 

»  NCOIC  -[www.ncoic.org] 

-  NCOIC  Interoperability  Framework  (NIF)  WG 

•  NIF  defines  the  applications,  data,  and  communications  elements 
required  to  design  and  evaluate  Network-Centric  Systems  with 
respect  to  interoperability 

-  NetCentric  Analysis  Tool  (NCAT)  WG 

-  Services  and  Information  Interoperability  WG 

-  Others 
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Agility 


21st  Century  Security  Challenges  characterized  by 
huge  amounts  of  uncertainty  and  risk 

Agility  is  the  answer  to  uncertainty  and  risk 


Robust  -  effective  across  a  range 
of  conditions; 

Resilient  -  able  to  function  / 
degrade  gracefully  /  reconstitute 
when  damaged 

Responsive  -  speed  of  recognition 
and  action; 

Flexible  -  multiple  ways  to 
succeed,  seamless  shifting; 
Innovative  -  learning  and  solving 
Adaptive  -  alteration  in  C2 
organization  and  process. 


Responsive 
Robust  Innovative 


Flexible 


Adaptive 


Resilient 
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Summary 


Challenges  to  Integration  of  FoS  into  SOS  architectures 

-  Complexity 

-  Dependency 

-  Emergent  Behavior  (tradeoff  flexibility  and  compose-ability 
versus  predictability) 

-  Collaboration 

Web  Services  and  SOA  are  not  the  only  solution 

-  (e.g.,  some  Sensor  to  Shooter  pairings) 

The  key  to  implementation  success 

-  New  and  evolved  services  must  be  easy  to  use  and  very  quick 
to  train  -  change  is  a  constant  in  this  equation 

-  Quickly  discoverable  services  on  the  GIG  -  the  Operator  will 
require  time-sensitive  information  superiority  on  the 
battlefields  of  the  future 


Goal:  Embrace,  Manage,  and  Hide  Complexity  of  SoS  - 
Maximize  Flexibility  and  Ease  of  Use  for  the  User 
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System 

Engineering 

Metrics 
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Why  Measure  Systems  Engineering? 


•  When  performance  is  measured  ... 
performance  improves 

•  When  performance  is  measured  and 
reported  ...  the  rate  of  performance  improves 

•  When  performance  is  measured,  reported, 
and  compared  ...  the  rate  of  performance  i 
continues  to  improve 


Problem 


•  Sys  Eng  Scope  is  Huge,  So  ... 

-  What  tenets  should  be  measured? 

-  What  are  the  key  characteristics? 

-  How  can  it  apply  across  different  programs  and 
organizations? 

•  Sys  Eng  Important,  But ... 

-  No  accepted,  standard  metrics 

-  No  measure  of  sys  eng  current  status 

-  No  metrics  for  both  PM  and  upper  management 


Sys  Eng  Metrics  Key  Characteristics 


•  Must  Measure  Major  Components  of  Sys  Eng 

•  Must  Be  Targeted  for  Management 

•  Must  Be  Few  in  Number 

•  Must  Describe  Current  Status,  Not  Lagging 

•  Must  Allow  For  Comparison  Between  Programs, 
Organizations,  and  Time 

•  Must  Be  Cumulative  (Ability  to  Roll-Up) 

•  Must  Avoid  Extensive  Data  Collection  Efforts 


Solution:  Sys  Eng  “Dashboard” 


•  Measure  Five  Key  Areas  of  Sys  Eng: 

-  Requirements  Management 

-  Risk  Management 

-  Incentivizing  Contractors 

-  Robustness/LCC 

-  Process  Management 

•  Used  on  All  Programs 

•  Regularly  Shown  at  Organization  Staff  Meetings 


1.  Requirements  Management  Metric 


*  Most  Important  Area 

*  Quantify,  quantify,  quantify 

*  Level  of  Detail 

-  Appropriate  to  Life  Cycle 

-  Examples 

*  Objective  Review 

*  Agreement  &  Understanding 

-  User 

-  Contractor 

-  Program  Manager 

*  Sources 


•  • 


Requirements  Management  Metric 


Requirements 

Reviewed 

Requirements 

Quantified 

^  Requirements 
Changed 


2.  Risk  Management  Metric 


•  Proactive 

•  Dynamic 

•  Reviewed  Regularly 

•  Tangible  Reduction  Plan 

•  Tracked 


Basic  Risk  Rating  Chart 


73 

O 
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RISK  ASSESSMENT 


HIGH  -  Unacceptable. 
Major  disruption  likely. 
Different  approach 
required. 

MODERATE  -  Some 
disruption.  Different 
approach  may  be  required. 


LOW  -  Minimum  impact. 
Minimum  oversight  needed 
to  ensure  risk  remains  low. 


Low  Med.  High 


Consequence 


Risk  Assessment  Metric 


Consequence 


Risk  Management  Metric 


Low  Med.  High 

Consequence 


%  With  Plan 


50% 


30% 


3.  Robustness/LCC  Metric 


*  Hard  to  Measure 

*  Measures  More  the  “Attempt”  or  Effort 

*  Can  Include  Underlying  Processes 

-  Example:  Type  of  paint  or  the  paint  application  process 

*  Need  “Toolbox”  Vice  One  Approved  Way 

-  Lean  processes 

-  Trade  studies 

-  Benchmarks 

-  Combining  components 

-  COTS 

-  Paredo  Charts 

-  Etc. 


Components 


Robustness/  LCC  Metric 


Time 


4.  Incentivizing  Contractors  Metric 


*  Required  for  USAF  by  Policy 

-  Policy  Memo  03A-005,  9  Apr  03 

-  Subject:  “Incentivizing  Contractors  for  Better  Systems 
Engineering” 

-  Signed  by  Marvin  R.  Sambour,  Assistant  Secretary  of  the  Air 
Force  (Acquisition) 

*  “A  more  robust  SE  environment  can  only  be  achieved 
through  joint  cooperative  efforts  with  our  contractors.” 


“...incentivize  your  contractors  to  perform  robust  SE...” 


Incentivizing  Contractors  Metric 


%  of  Contracts  with 
Sys  Eng  Incentives 


5.  Process  Management  Metric 


•  List  Program’s  Key  Processes 

-  Configuration  Management 

-  Waivers 

-  Quality 

-  Aircraft  Structural  Integrity  Program 

-  Deficiency  Reviews 

-  Etc. 

•  Each  Program  Does  Own  Processes 

•  For  Each  Process,  4  “Steps” 

-  Define  &  Document 

-  Lean,  Improve  or  Refine 

-  Keep  Current  by  Periodic  Reviews 

-  Measure  the  Process 


Process  Management  Metric 
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Current 

Lean/Improve 

Documented/ 
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Not  Done 
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Program  Sys  Eng  Dashboard 


•  Developed  Individual  Metrics  for  the  Five  Key 
Areas  of  Systems  Engineering: 

-  Requirements  Management 

-  Risk  Management 

-  Incentivizing  Contractors 

-  Robustness/LCC 

-  Process  Management 

•  Now  Put  it  All  Together  For  the  Proposed 
Program’s  Sys  Eng  Dashboard... 


Program  Sys  Eng  Dashboard 


Med. 


Low 


Requirements 


0/1 


2/3 


Low 


Med. 

Risk 


Contracts 


Doc. 


Processes 


How  to  Roll-Up  from  Program  to  Organization 


*  Requirements  Management 

-  Convert  each  program  to  a  percentage 

-  Display  average  (each  program  has  equal  weight) 

*  Risk  Management 

-  Convert  each  program  “square”  to  percentage 

-  Display  average  “square’s”  percentage  (equal  weight) 

*  Incentivizing  Contractors 

-  Bottom  number  equals  sum  of  contracts 

-  Depict  percentage  of  contracts  (program  independent) 

*  Robustness/LCC 

-  Calculate  reveiwed/changed  as  a  percentage 

-  Display  avg  percentage  (equal  weight) 

*  Process  Management 

-  Depict  overall  percentage  for  each  category  (process/program 
independent) 


Organization  Requirements  Metric  (%) 


Requirements 

Reviewed 

Requirements 

Quantified 

Requirements 

Changed 


Likelihood 


Organization  Risk  Metric  (%) 
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Organization  Requirements  Metric  (%) 


Organization  Incentivizing  Contractors  Metric 


%  of  Contracts  with 
Sys  Eng  Incentives 


Organization  Process  Metric  (%) 


Measured 

Current 

Lean/Improve 

Documented/ 

Defined 


1 

100% 


Organization  Sys  Eng  Dashboard 


Requirements 


Reviewed 


Changed 


LCC/Robust 


Contracts  Doc. 


Processes 


Summary 


*  Sys  Eng  Important,  but  No  Consistent  Way  to 
Measure. ..Until  Now 

*  Need  Concurrent  Metrics. ..Not  Lagging 

*  Metrics  For  Management... Essential  to  Drive  Action 

*  What  to  Measure... Sys  Eng  “Dashboard” 

*  Means  To  Use. ..Regular  Part  of  an  Organization’s 
Overall  Management  Indicators 


Allows  Comparison. ..Drives 
Improvement 


wi 


Questions? 


Sample:  5  -  Level  Risk  Rating  Chart 


RISK  ASSESSMENT 


1 

Minimal  or  no  impact 

Minimal  or  no  impact 

Minimal  or  no  impact 

None 

2 

Acceptable  with  some 
reduction  in  margin 

Additional  resources  required; 
able  to  meet  need  dates 

<5% 

Some  impact 

3 

Acceptable  with 
significant  reduction 
in  margin 

Minor  slip  in  key  milestone; 
not  able  to  meet  need  dates 

5  -  7% 

Moderate  impact 

4 

Acceptable,  no 
remaining  margin 

Major  slip  in  key  milestone 
or  critical  path  impacted 

>7-10% 

Major  impact 

5 

Unacceptable 

Can’t  achieve  key  team  or 
major  program  milestone 

>  10% 

Unacceptable 

HIGH  -  Unacceptable. 

Major  disruption  likely. 
Different  approach  required. 
Priority  management 
attention  required. 

■  MODERATE  -  Some 
disruption.  Different 
approach  may  be  required. 
Additional  management 
attention  may  be  needed. 

■  LOW  -  Minimum  impact. 
Minimum  oversight  needed 
to  ensure  risk  remains  low. 


and/or  Impact  on  Other  Teams 


ASSESSMENT  GUIDE 


LIKELIHOOD : 
Level  What 


What  Is  The  Likelihood 
The  Risk  Will  Happen? 

Remote 

Unlikely 

Likely 

Highly  Likely 
Near  Certainty 
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CONSEQUENCE : 

Given  The  Risk  Event  is  Realized,  What  is  the  Magnitude  of  the  Impact? 


Technical 

Level  Performance 


and/or  Schedule 


Risk  Rating 


Risk  Handling  Plan  -  “Waterfall” 


High 


Medium 


Low 


Time 


Converting  High-Level 
Systems  Engineering 
Policy  to  a  Workable 
Program 


26  Oct  05 


James  C.  Miller 

Chief  Engineer 

327th  CLSG 

Phone:  736-4294 

james.c.miller@tinker.af.mil 


Background 


•  Prior  to  1997,  numerous  incidents,  mishaps  and 
configuration  occurred  in  the  Air  Force  (AF) 

•  AF  recognized  need  for  a  disciplined  technical 
process  for  the  development  and  sustainment  of 
AF  systems 

•  In  1997,  AF  instituted  the  Operational  Safety, 
Suitability  and  Effectiveness  (OSS&E)  Program 

•  OSS&E  Focused  on  sustainment  due  to  trend  in 
field  support  process  deficiencies 


Background  (Cont) 


•  OSS&E  mandated  6  levels  for  certification 

-  Included  milestones,  metrics,  and 
entry/exit  criteria  for  each  level 

•  Implemented  throughout  the  AF 

-  Certification  of  Level  6  required  by  Oct  05 

•  Good  effort,  supported  by  most  Chief  Engineers 

•  However,  OSS&E  is  a  subset  of  systems 
engineering 

•  Over  last  2  years,  AF  started  releasing  high-level 
policy  regarding  systems  engineering 


SE  Policy  Addendum 

Signedb^^i^MarviriR^amb^rj^^^^ecA^Acq^^^on)Api^3&Jar^4 


•  Policy  Memo  03A-005,  9  Apr  03 

-  Subj:  Incentivizing  contractors  for  Better  Systems 
Engineering 

-  “An  immediate  transformation  imperative  for  all  our 
programs  is  to  focus  more  attention  on  the  application 
of  Systems  Engineering  principles...” 

-  Directing  the  following: 

•  A.  Assess  ability  to  incentivize  contractors  to  perform 
robust  SE 

•  B.  Develop  SE  performance  incentives 

•  C.  Include  SE  processes/practices  during  all  program 
reviews 

•  Policy  Memo  04A-001,  7  Jan  04 

-  Subj:  Revitalizing  Air  Force  and  Industry  Systems 
Engineering  (SE)  -  Increment  2 

-  “...intended  to  institionalize  key  attributes  of  an 
acceptable  SE  approach  and  outcome...” 

-  “...must  focus  on  an  end  state...” 


Systems  Engineering  Policy  in  DoD 

Signe^by_theJHonojgbj^Mjk^Wynne,JJSD(AT&L)  (Acting)f  eb_20,  2004 


*  All  programs,  regardless  of  ACAT  shall: 

-  Apply  an  SE  approach 

-  Develop  a  Systems  Engineering  Plan  (SEP) 

•  Describe  technical  approach,  including  processes, 
resources,  and  metrics 

*  Detail  timing  and  conduct  of  SE  technical  reviews 

*  Director,  DS  tasked  to  provide  SEP  guidance  for 
DoDI  5000.2 

-  Recommend  changes  in  Defense  SE 

-  Establish  a  senior-level  SE  forum 

-  Assess  SEP  and  program  readiness  to  proceed  before 
each  DAB  and  other  USD(AT&L)-led  acquisition  reviews 


SEP  Implementation  Guidance 

^^Pei^USD(AT&L)Defens^System^Mem^igne^Mai^0^^04^^ 

*  Submitted  to  MDA  at  each  Milestone,  SEP 
describes: 

-  Systems  engineering  approach 

•  Specific  processes  and  their  tailoring  by  phase 

•  Both  PMO  and  Contractor  processes 

-  Systems  technical  baseline  approach 

•  Use  as  control  mechanism,  including  TPMs  and 
metrics 

-  Technical  review  criteria  and  outcomes 

•  Event  driven 

•  Mechanism  for  assessing  technical  maturity  and  risk 

-  Integration  of  SE  with  IPTs  and  schedules 

•  Organization,  tools,  resources,  staffing,  metrics, 
mechanisms 

•  Integrated  schedules  (e.g.,  IMP  and  IMS) 


SE  Policy  Addendum 

Signe^b^heh^Torabl^^^^ynnejUSD(A^L)(Actinc|)Oc^^^004 


*  Each  Program  Executive  Officer  (PEO)  shall  have  a 

lead  or  chief  systems  engineer 

*  The  PEO  lead  or  chief  systems  engineer  shall: 

-  Review  assigned  programs’  SEPs  and  oversee  their 
implementation 

-  Assess  the  performance  of  subordinate  lead  or  chief 
systems  engineers 

*  Technical  reviews  shall: 

-  Be  event  driven  (vice  schedule  driven) 

-  Conducted  when  the  system  under  review  meets  review 
entrance  criteria  as  documented  in  the  SEP 

-  Include  participation  by  subject  matter  experts  independent 
of  the  program,  unless  waived  by  SEP  approval  authority  in 
the  SEP 


Defense  Acquisition  Guidebook,  Chapter  4,  Section  4.2 


•  SE  terminology,  models,  and  standards 
-  Technical  Management  Processes 


•Decision  Analysis 

•Technical  Planning 

•Technical 

Assessment 

•Requirements  Mgmt 


•Risk  Management 

•Configuration  Mgmt 

•Technical  Data 
Mgmt 

•Interface 

Management 


-  Technical  Processes 


•Requirements 

•Integration 

Development 

•Verification 

•Logical  Analysis 

•Validation 

•Design  Solution 

•Transition 

•Implementation 

So  What  is  the  Problem? 


•  High-level  policy  is  a  good  and  necessary  first 
step,  however,  a  more  detailed  direction  is 
essential  to  turn  the  policy  into  a  workable, 
grass-roots  program 


So  What  Do  We  Do  About  It? 


•  Propose  a  step-by-step  approach  to  begin 
implementing  systems  engineering  throughout 
the  organization 

•  Is  a  tangible  approach  that  is: 

-  Aimed  at  the  working  level 

-  Affects  all  phases  of  a  program’s  lifecycle 

-  Applicable  throughout  entire  organization 

-  Accounts  for  organization’s  progress  through  metrics 

•  Approach  is  based  on  the  OSS&E  construct 


Summary  of  the  OSS&E  Construct 


•  Level  1  Criteria — Chief  Engineer  Assigned 

•  Level  2  Criteria — Configuration  Control 
Processes  Established 

•  Level  3  Criteria — Document  Plan  to  Assure  and 
Preserve  OSS&E  Baseline  Characteristics 

•  Level  4  Criteria — OSS&E  Baselines  Developed 
and  Coordinated  with  User 

•  Level  5  Criteria — OSS&E  Assessment  of  Fielded 
Systems,  Resolve  Disconnects  with  Baseline 

•  Level  6  Criteria — Monitor  and  Maintain  Full 
OSS&E  Policy  Compliance 


Notional  Sys  Eng  Implementation  Phases 


•  Phase  1: 

•  Phase  2: 

•  Phase  3: 

•  Phase  4: 

•  Phase  5: 

•  Phase  6: 

•  Phase  7: 


Awareness  of  Need 
Workforce  Training/Education 
Identify  Applicable  Programs/Orgs 
Identify  and  Define  Processes 
Incentivize  Contractors/Partners 
Develop  Library  of  Tools 
Track  Progress  via  Metrics 


Phase  1 :  Awareness  of  Need 


*  Phase  1  Taskings: 

-  Identify  Focal  Point  for  SE  policy,  practice  and  implementation 

-  Brief  senior  leaders  on  SE  Defintion,  SE  policy,  and  SE 
“reinvigoration”  plan 

-  Develop  “Road  Show”  for  subordinate  offices  and/or  programs 

*  Exit  Criteria: 

-  Focal  Point  identified  and  appointed 

-  Senior  leaders  briefed  with  documented  support/concurrence 

-  Road  show  presented  to  all  applicable  offices/programs 


Phase  2:  Workforce  Training/Education 


•  Phase  2  Taskings: 

-  Define  minimum  training/certification  requirements 

-  Train  working  level  engineers 

-  Train  program  managers 

-  Train  Lead/Chief  Engineers 
and  Directors  of  Engineering 

•  Exit  Criteria: 

-  80%  of  working  level  engineers  trained 

-  95%  of  program  managers  trained 

-  100%  of  Lead/Chief  Engineers,  and  Directors  of 
Engineering  trained 


Phase  3  Taskings: 

-  List  all  applicable  Programs/Organizations,  such  as: 

•  All  OSS&E  identified  programs 

•  Other  major  progams  and  projects 

•  Engineering  Contracts 

•  Technology  Insertion  Projects 

•  Relevant  functional  offices  (Engineering,  Logistics. 

-  Notify  each  affected  program  and  organization 

-  May  do  incrementally,  but  if  so,  build  schedule 
Exit  Criteria: 

-  Documented  process  to  identify  programs/orgs 

-  Clear,  comprehensive  list 

-  Schedule  phase  due  dates  for  all 
programs/organizations 


Phase  4:  Identify  and  Define  Processes 


•  Phase  4  Taskings: 

-  Develop  list  of  applicable  common  processes 

•  At  a  minimum  include: 

-  Requirements  Management 

-  Risk  Management 

-  Configuration  Management 

-  Test  Management 

-  Life  Cycle  Cost/Robustness 

-  Define/standardize  each  process 

•  Use  best  practices 

•  Clearly  document  each  process 

-  Systems  Engineering  Plan  (SEP) 

•  Exit  Criteria: 

-  List  of  common,  documented  processes 


Phase  5:  Incentivize  Contractors/Partners 


Phase  5  Taskings: 

-  Devise  selection  criteria 

-  List  applicable  contracts 

-  Develop  tailorable  “template” 

-  Ensure  language  in  contracts 

-  Determine  how  to  verify  SE  compliance 

Exit  Criteria: 


-  List  of  all  targeted  contracts 

-  SE  an  incentivized  factor  in  all  applicable  contracts 

*  Given  the  nature  of  contracts,  this  can  be  a  sliding  scale, 
e.g  25%  in  FY06,  50%  by  2007,  etc... 


Phase  6:  Develop  Library  of  Tools 


Phase  6  Taskings 

-  Define  “How  To”  and  examples  for: 

•  Risk  Management 

•  Requirement  Management 

•  Configuration  Management 

•  Designing  for  Life  Cycle  Cost 

•  Others 


Phase  7:  Track  Progress  via  Metrics 


•  Phase  7  Taskings 

-  Develop  a  set  metrics 

-  Determine  when  and  at  what  level(s)  they  will 
be  regularly  briefed 

•  Exit  Criteria 

-  Developed  set  of  metrics 

-  Metrics  displayed  regularly  at  staff  meetings 


Reviewed 


Sample  Program  Sys  Eng  “Dashboard” 


obust 


Sample  Organization  Sys  Eng  “Dashboard” 


Requirements  (%) 


Reviewed 


Changed 


LCC/Robust  (%) 


Contracts  Doc. 


Processes  (%) 


AF  is  releasing  necessary  high-level  policy  regarding  SE 
Need  a  workable  grass-roots  means  to  implement  SE 
Developed  a  notional  7  phase  approach 

-  Similar  to  OSS&E  construct 

-  Aimed  at  the  working  level 

-  Affects  entire  lifecycle 

-  Applicable  to  whole  organization 

-  Accounts  for  progress  i 


Provides  a  concrete,  tangible  starting  point  to  help  first 
line  supervisors  and  working  engineers  begin 
implementing  systems  engineering 


wi 


Questions? 


OSS&E-  Level  1 


•  Level  1  Criteria — Chief  Engineer  Assigned 

•  Exit  Criteria 

-  System/End-Item  (S&EI)  on  OSS&E  S&EI  List 

-  Chief  Engineer  identified  on  OSS&E  S&EI  list 

-  Process  is  in  place  to  update  S&EI  list  (1.1.1  a) 


OSS&E-  Level  2 


•  Level  2  Criteria — Configuration  Control 

Processes  Established 

•  Exit  Criteria 

-  Configuration  control  processes  identified  and 
documented  at  the  program  level 

-  Configuration  control  process  training  requirements 
identified 

-  Configuration  control  processes  are  in-place  and 
operating 

-  Delegated  authority  identified  and  documented 


OSS&E-  Level  3 


•  Level  3  Criteria — Plan  to  Assure  and  Preserve 
OSS&E  Documented 

•  Exit  Criteria 

-  Plan  shall  include  strategies/approach  for: 

•  Identifying,  reconciling,  and  preserving  OSS&E  baseline 
characteristics 

•  Achieving  and/or  maintaining  required  certifications 

•  Establishing  OSS&E  program  level  and  product  line  metrics 

•  Identifying  data  system  feedback  mechanisms 

-  OSS&E  Execution  Plan  coordinated  with: 

•  Appropriate  Product,  Logistic,  Test,  and  Specialty  Centers 


OSS&E-  Level  4 


•  Level  4  Criteria — OSS&E  Baselines  Developed 
and  Coordinated  with  User 

•  Exit  Criteria 

-  OSS&E  baseline  characteristics  identified 

-  Critical  Characteristics  for  measuring  safety, 
suitability,  and  effectiveness  selected 

-  OSS&E  baseline  characteristics  and  metrics 
coordinated  with  users 


OSS&E-  Level  5 


•  Level  5  Criteria — OSS&E  Assessment  of  Fielded 
Systems/End-Items 

•  Exit  Criteria 

-  Fielded  system/end-item  data  gathered 

-  OSS&E  baseline  characteristics  assessment 
completed 

-  OSS&E  baseline  disconnects  identified 

-  Recommended  corrective  actions  to  users 


OSS&E-  Level  6 


•  Level  6  Criteria — Full  OSS&E  Policy  Compliance 

•  Exit  Criteria 

-  Level  5  corrective  actions  completed 

-  All  required  certifications  in  place  and  maintained 

-  Metrics  and  feedback  systems  monitoring  OSS&E 
health 

-  Processes  established  and  in  place  to  maintain 
OSS&E  baseline  characteristics 


8th  Annual  Systems  Engineering  Conference 

Sponsored  by  the 

National  Defense  Industrial  Association 

San  Diego  CA 
October  2005 
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Integrating  MIL-STD-882  System  Safety  Products 
Into  The  Concurrent  Engineering  Approach  To 
System  Design,  Build,  Test,  And  Delivery  Of 
Submarine  Systems  At  Electric  Boat. 


Ricky  Milnarik 


GENERAL  DYNAMIC 

Electric  Boat 


SEC-8  2 


Introduction 


Electric  Boat  has  been  building  submarines  for  the 
U.  S.  Navy  for  over  100  years. 


In  1900  Electric  Boat 
delivered  the 
U.  S.  Navy’s  first 
submarine,  the 
USS  Holland. 
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Introduction 


Subsequent  to  the  USS  Holland,  Electric  Boat  has 
delivered  over  270  submarines  to  the  U.S.  Navy. 
In  October  2004  the  USS  VIRGINIA,  the  first  ship  in 

a  new  class 

of  fast  attack 

submarines, 

was  delivered 

the  U.  S.  Navy. 


GENERAL  DYNAMIC 

Electric  Boat 


SEC-8  4 


Introduction 


The  VIRGINIA  Class  Submarine  is  the  first  class  of 
submarine  built  at  Electric  Boat  that  uses  the 
Integrated  Product  and  Process  Development 
(IPPD)  process  to  conduct,  manage  and  status 
the  ship  design,  ship  construction  and  life  cycle 
support. 

The  IPPD  process  is  a  dynamic  concurrent 

engineering  concept  that  includes  integration  of 
system  safety  engineers  into  design/  build 
teams  (DBT). 
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Introduction 


Before  the  IPPD  process  was  implemented  a  serial 
approach  to  submarine  design-to-construction  was 
taken. 

Upon  Navy  approval  of  the  drawings  a  full  scale 
wooden  mockup  of  the  lead  ship  was  built  and 
maintained. 

The  dynamics  of  the  design/build  team  concept  is 
made  possible  through  the  use  of  the  Computer 
Aided  Three-Dimensional  Interactive  Application 
(CATIA)  software  design  tool  to  develop  electronic 
mockups  in  place  of  building  wooden  mockups. 
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Introduction 


The  design/build  team  concept  also  necessitated 
tailoring  how  traditional  MIL-STD-882  system 
safety  program  products  were  developed  and 
used  to  provide  a  complete  evaluation  of  the 
system(s)  under  development. 
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Integrated  Product  and  Process 

Development 


The  basis  for  IPPD  is  the  design-to-build 
approach. 

This  methodology  consists  of  activity-based 
product  management  and  concurrent 
engineering  DBTs. 

Team  assignments  are  structured  in  accordance 
with  program  development  and 

manufacturing  needs. 
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Integrated  Product  and  Process 

Development 


Ensures  that  all  requirements  of  conceptual 
engineering,  design,  fabrication,  assembly, 
and  test,  that  support  system  safety  are 
evaluated  and  analyzed  early  in  the 
acqusition  process. 
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Integrated  Product  and  Process 

Development 


Concept  To  Delivery 


► 


IPPD  Team  Staffing 
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Design  /  Build  Teams 


Design  Build  Teams  consist  of: 

-  Program  Management  Teams 

-  Functional  Area  Teams 

-  System  Integration  Teams  (SIT) 


GENERAL  DYNAMIC 
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Design  /  Build  Teams 


DBT  functional  managers  /  technical  leaders 
have  direct  management  and  control  of  their 

specific  functional  areas. 


FUNCTIONAL  AREA  TEAM  SYSTEM  INTEGRATION  TEAMS 
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Design  /  Build  Teams 


DBTs  also  manage  both  technology  and 

program  development  and  exercise  authority 
in  ensuring  component  and  system  integrity 
via  technical  design  reviews  and  approval 
circuits. 

This  responsibility  broadens  the  awareness  and 
involvement  of  team  members  and  creates  a 
sense  of  ownership  of  the  design  efforts  and 
system  safety  products. 
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Design  /  Build  Teams 


DBTs  are  made  up  of  representatives  from 
Electric  Boat,  government  suppliers, 
government  laboratory  personnel,  Navy 
operators,  independent  government 
review/certification  board  members  (e.  g. 
Weapon  System  Explosives  Safety  Review 
Board,  SUBSAFE  ,  Deep  Submergence 
System  (diver  safety)  etc.)  and  teaming 
shipyards. 
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Design  /  Build  Teams 


A  typical  DBT  makeup  is  shown  below 


Design 


Operations 


Materials 


Engineering 

System  Safety 


Planning 

NAVSEA 


Purchasing  Quality  Assurance, 

Navy  Operators 
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System  Integration  Teams 


System  Integration  Teams  (SITs)  develop, 
integrate,  and  optimize  systems  in  the  ship 
and  prepare  technical  deliverables  by: 

Developing  and  evaluating  system  concepts  and 
new  components,  conducting  trade-off  studies, 
developing  system  diagrams,  class  drawings, 
component  specifications  etc. 

Performing  safety  analyses  on  new  and 
significantly  modified  legacy  ship  systems  and 
components  in  accordance  with  the  System  Safety 
Program  Plan. 
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System  Integration  Teams 


Establishing  technical  interfaces  with  government 
agencies,  laboratories,  and  other  contractors. 

Integrating  discipline-specific  individuals  and 
individuals  with  appropriate  specialty 
expertise  (e.g.  system  safety  engineers, 
production,  finance,  integrated  logistics 
support  environmental  compliance  etc.). 
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System  Integration  Teams 


Typical  Submarine  Systems 

Torpedo  Ejection 

Trim  and  Drain 

Propulsion  Plant 

Vertical  Launch 

Low  Pressure  Air 

High  Pressure  Air 

Weapons  Handling 

Main  Hydraulic 

Main  Seawater 

Communications  (Radio) 

HVAC 

Ships  Entertainment 

Combat  Control  Subsystem 

External  Hydraulic 

AC  Power/Interior 

Combat  Launch  Control 

Ship  Control 

Masts  and  Antennas 

Navigation 

Fresh  Water 

Atmosphere  Monitoring 

Sonar 

AC  Electrical  Power 

Interior  Communication 

Total  Ship  Monitoring 

DC  Electrical  Power 

Auxiliary  Seawater 

Non-Tactical  Data  Processing 

Lighting 

Main  Ballast  Tank  Low 

Escape  and  Rescue 

Fire  Fighting 

Pressure  Blow 
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System  Safety  Process 


Tailoring  of  the  system  safety  process  centered 
around: 

-  Formalized  SIT  meetings. 

-  Conduct  of  safety  hazard  analyses  as  a  team 
product. 

-  Use  of  CATIA  for  safety  hazard  analyses  and  Human 
Systems  Integration  (HSI)  into  design  products. 
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System  Safety  Process 


SIT  Meetings 

Since  the  SITs  contain  all  the  key  players  and 
decision  makers  for  the  system  under 
development.  Each  SIT  meeting: 

-  doubles  as  a  safety  working  group  meeting 

-  documents  system  and  safety  design  decisions 

-  documents  unresolved  issues  and  assigns  action 
items 

-  is  documented  on  official  minutes  to  ensure  continuity 
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System  Safety  Process 


Safety  Hazard  Analyses 

Traditional  MIL-STD-882  system  safety  tasks 
were  used  to  identify  potential  hazards. 

-  Preliminary  Hazard  Analyses 

-  Safety  Requirements  Analyses 

-  Software  Analyses 

-  Subsystem  Hazard  Analyses 

-  System  Hazard  Analyses 

-  Operating  and  Support  Hazard  Analyses 
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System  Safety  Process 


Safety  Hazard  Analyses  (cont’d) 

Because  of  the  dynamics  of  the  DBT  process  it 
was  decided  that  updating  previously 
completed  hazard  analyses,  when  additional 
information  became  available,  was  not 
feasible. 

Instead  each  completed  hazard  analysis 
portrayed  a  snap  shot  in  time  of  the  system 
under  evaluation. 
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System  Safety  Process 


Safety  Hazard  Analyses  (cont’d) 

Each  subsequent  hazard  analysis  built  upon  the 
previous  analysis  conducted. 

Significant  design  changes  or  identification  of 
new  hazards  that  came  up  between  hazard 
analyses  were  documented  on  an  Analysis 
Completion  Summary  (ACS)  Report  for 
continuity. 
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System  Safety  Process 

ANALYSIS  COMPLETION  SUMMARY  I 

System: _  Cognizant  Engineer: _ 

Date  Initiated: _  Date  Completed: _ 

Enclosures: 


Analysis  Summary: 

SAMPLE 


SAFETY  ANALYSIS  WORKSHEETS  (attached) 

1. _  2. _ 

3. _  4. _ 


Safety  Engineer: 
Team  Leader: 
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System  Safety  Process 


Safety  Hazard  Analyses  (cont’d) 


Concept  To  Delivery 

- ► 


Operations  Organization 
(Shipyard  Construction  Safety) 


Concept  and  Design 
Development 

Manufacturing  Construction  &  Test  1  Assembly,  Installation,  Test 

&  Test  1  1  &  Delivery 

System  Safety  or  ESOH 
Program  Plan 

Preliminary  Hazard  Analysis 
Requirements  Hazard 
Analysis 

Subsystem  Hazard  Analysis 

System  Hazard  Analysis  1  Operating  &  Support  Hazard  Analysis 

1  Safety  Assessment  Report 

1 

1 

1 

1 

B  Hazard  Tracking  and.Risk  Resolution 

Design  Development  and  System  Safety  Products 
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System  Safety  Process 


Safety  Hazard  Analyses  (cont’d) 

Provide  System  Safety  Objective  Quality  Evidence 
for  the  systems  under  development: 

-  Completed  safety  hazard  analyses 

•  Analysis  Completion  Summary  Reports 

-  SIT  meeting  minutes 

-  Program  design  review  findings 

-  Independent  government  review  board  findings 

•  Weapon  System  Explosives  Safety  Review  Board 

-  Hazard  closure  forms 


GENERAL  DYNAMIC 

Electric  Boat 
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System  Safety  Process 


CATIA  Program 

Electronic  design  data  created  in  CATIA  is 
controlled  and  stored  in  the  CATIA  Data 
Manager  as  the  central  repository  that  supports 
the  various  elements  of  the  IPPD  process. 

CATIA  displays  were  projected  on  screens  in 
Electronic  Visualization  Simulation  (EVS)  rooms 
during  SIT  meetings  allowing  SIT  members  to 
view  the  latest  system  design  and 
arrangements. 
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Electric  Boat 
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System  Safety  Process 


CATIA  Program  (cont’d) 

Examples  of  HSI  efforts  through  the  use  of  CATIA 
were: 

-  Reserving  pull-spaces  on  drawings  for  racking  out 
equipment  during  maintenance. 

-  Readily  identifying  interference  with  other 
systems/subsystems/equipment. 

-  Demonstrating  critical  equipment  removal  and 
replacement  flow-paths. 

-  Reserving  spaces  on  drawings  for  access  to  vital 
equipment  (safety  of  ship). 
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System  Safety  Process 


CATIA  Program  (cont’d) 


Ergo  Man 

Representing  fifth 
through  ninety-fifth 
percentile  body 
dimensions)  used  to 
evaluate  system 
design  in  terms  of 
whole-body  fit, 
access/emergency 
egress,  reach  and 
visual  field  etc. 


SSGN  Lockout  Chamber 
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System  Safety  Process 


CATIA  Program  (cont’d) 

Through  the  use  of  CATIA,  system  safety  engineers 
identified  HSI  issues  early  and  throughout  the 
design  phase. 

Eliminating  the  need  for  separate  operator  and  maintainer 
human  engineering  analyses. 

Unresolved  HSI  issues  were  documented  in 
applicable  hazard  analyses  or  analysis 
completion  summary  reports. 


GENERAL  DYNAMIC 
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Lessons  Learned 


The  IPPD  process  not  readily  accepted  by  all  DBT 
members  e.g.,  contractors,  subcontractors, 
government  agencies  not  using  or  familiar  with 
the  design  build  team  process. 

The  IPPD  process  only  as  good  as  the  DBT  training 
provided  to  team  members. 
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Lessons  Learned 


The  IPPD  process  resulted  in  a  lower  number  of 
documented  hazards  measured  against 
traditional  system  safety  processes  (metrics, 
added  value  of  a  system  safety  program) 
because  most  hazards  were  designed  out  during 
the  SIT  meetings. 

DBT  members  treated  system  safety  engineers  as 
partners  rather  than  “safety  police”. 
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Tom  Schiller 
Andrew  Levine 

Naval  Surface  Warfare  Center,  Carderock  Division 

William  H.  Mish  Jr 
M.  Rosenblatt  and  Son  an  AMSEC  LLC  Company 


MBMU 


Stremgth  through  Industry  Technology 


INTRODUCTION 


DoD  POLICIES  and  DIRECTIVES 

-  Application  of  new  DoD  5000  series  and  Joint 
Integration  and  Development  Systems  (JCIDs)  to  ship 
acquisition  programs  through  the  implementation  of  the 
Open  Systems  Joint  Task  Force  (OSJTF)  Modular  Open 
Systems  Approach  (MOSA) 

CHANGING  THREATS 

-  New  mission  capabilities 

-  Technology  refresh  to  adapt  to  changing  world  climate 

-  Requires  rapid  system  and  component  change-out 
FLEXIBLE  FORCE  -  MODULAR  ADAPTABLE  FLEET 

-  Allows  for  rapid  change  of  a  multi-mission  ship 

-  Allows  a  single  ship  to  have  multiple  capabilities  to 
support  and  defend  against  air,  surface  and 
submersibles  assets 

AFFORDABLE  FLEET  -  FAMILY  OF  SHIPS 

-  Allows  for  cross-platform  component  commonality  and 
interchangeabi  ity  between  ships  and  ship  designs 
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DOD  DIRECTIVE  5000.1 
THE  DEFENSE  ACQUISITION  SYSTEM 


“Acquisition  programs  shall 
be  managed  through  the 
application  of  a  systems 
engineering  approach 
that  optimizes  total  system 
performance  and 
minimizes  total  ownership 
costs.  A  modular,  open- 
systems  approach  shall  be 
employed,  where 
feasible.” 


Department  of  Defense 

DIRECTIVE 

NUMBER  5000.1 
May  12,  2003 
Certified  Current  as  of  November  24,  2003 

USD(AT&L) 

SUBJECT :  The  Defense  Acquisition  System 

References:  (a)  DoD  Directive  5000.1,  "The  Defense  Acquisition  System,"  October  23, 
2000  (hereby  canceled) 

(b)  DoD  Instruction  5000.2.  "Operation  of  the  Defense  Acquisition  System," 
May  12,  2003 

(c)  DoD  5025. 1-M.  "DoD  Directives  System  Procedures,"  current  edition 

(d)  Title  10,  United  States  Code,  "Armed  Forces" 

(e)  Section  2350a  of  title  10,  United  States  Code,  "Cooperative  Research  and 
Development  Projects:  Allied  Countries" 

(f)  Section  2751  of  title  22,  United  States  Code,  "Need  for  international 
defense  cooperation  and  military  export  controls;  Presidential  waiver; 
report  to  Congress;  arms  sales  policy" 

(g)  Section  2531  of  title  10,  United  States  Code,  "Defense  memoranda  of 
understanding  and  related  agreements" 

(h)  Federal  Acquisition  Regulation  (FAR),  current  edition 

(i)  Section  1004,  Public  Law  107-314,  "Bob  Stump  National  Defense 
Authorization  Act  for  Fiscal  Year  2003,"  "Development  and 
Implementation  of  Financial  Management  Enterprise  Architecture" 

(j)  DoD  Directive  8500.1.  "Information  Assurance  (IA),"  October  24,  2002 

(k)  DoD  Directive  4630.5.  "Interoperability  and  Supportability  of 
Information  Technology  (IT)  and  National  Security  Systems  (NSS)," 
January  11,  2002 

(l)  DoD  Directive  2060.1,  "Implementation  of,  and  Compliance  with,  Arms 
Control  Agreements,"  January  9,  2001 


1.  PURPOSE 


This  Directive: 


1.1.  Reissues  reference  la)  and  authorizes  publication  of  reference  (b), 
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OPEN  SYSTEMS  JOINT  TASK  FORCE 

(OSJTF) 


"The  OSJTF’s  modular,  open 
systems  approach  is  a  key 
enabler  in  the 
Department's  focus  on 
joint  architectures  and 
evolutionary  approach  to 
weapon  systems 
acquisition.  All  acquisition 
programs  should  employ 
a  modular,  open  systems 
approach." 


FffQGflAM  MANAGER'S! 


A  Modular 
Open  Systems 
Approach  (MOSA) 
To  Acquisition 


OPEN  SYSTEMS 


JOINT  TASK  FORCF 


Folding  ntfaffiahl^ 
and  fntsrupmbk 
combat  capabilities 


www.  ncq.ovi.m  UA asjtf 
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MODULAR  OPEN  SYSTEMS  APPROACH 

(MOSA) 


Integrated  Business  and 
Technical  Strategy 


Vision 


cipies 


v"  Ease  of  Change 
v"  Reduced  Total  Ownership  Cost 
Reduced  Cycle-Time 


stabhsh  Enabling  Environment 


Employ  Modular  Design 
Designate  Key  interfaces 
Select  Open  Standards 


v  Enabling  Joint  Integrated 

Architectures  and  Interoperability 


Certify  Conformance 


v  Risk  Mitigation 
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MOSA  AS  AN  ENABLER 


•  The  MOSA  approach  is  an  enabler  to  achieve  the 
following  objectives: 

-  Adapt  to  evolving  requirements  and  threats 

-  Promote  transition  from  science  and  technology  into  acquisition 
and  deployment 

-  Facilitate  systems  integration 

-  Reduce  the  development  cycle  time  and  total  life-cycle  cost 

-  Ensure  that  the  system  will  be  fully  interoperable  with  all  the 
systems  which  it  must  interface,  without  major  modification  of 
existing  components 

-  Leverage  commercial  investment 

-  Enhance  access  to  cutting  edge  technologies  and  products 
from  multiple  suppliers 

-  Enhance  commonality  and  reuse  of  components  among 
systems 

-  Mitigate  the  risks  associated  with  technology  obsolescence 

-  Mitigate  the  risk  of  a  single  source  of  supply  over  the  life  of  a 
system 

-  Enhance  life-cycle  supportability 

-  Increase  competition 
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THE  NAVY’S  NEED  FOR  MODULES  AND 

OPEN  SYSTEMS 


“Controlling  cost  while  decreasing 
the  cycle  time  for  technology 
insertion  will  require  the  use  of 
open  architectures,  module 
interface  standards,  commercial 
processors,  etc.  in  conjunction  with 
strict  configuration  control.” 

Mr.  John  J.  Young,  Jr.,  Assistant  Secretary  of  the  Navy 
(Research,  Development,  and  Acquisition)  before  the 
procurement  subcommittee  of  the  house  armed  services 
committee  United  States  House  of  Representatives  Fiscal  Year 
2003  Navy/Marine  Corps  Shipbuilding  programs  March  20th 
2002. 

NDIA  Systems  Engineering  7 
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NAVY  OPEN  SYSTEMS  INITIATIVES 


•  Affordability  Through  Commonality  (ATC)  program 
transitioned  to  Total  Ship  Open  Systems  Architecture  (TOSA) 

•  TOSA  IPT  formed  in  1998 

-  Acquisition  reform  with  emphasis  on  “letting  Industry  do  it” 

-  Bring  Open  Systems  Architectures  (OSA)  concepts  to  ship 
design 

-  Reduce  the  Total  Ownership  Cost  (TOC)  of  ships 

•  Achieve  Fleet-Wide  commonality  through  maximum  use  of 
commercial  equipment  while  managing  risk 

•  Use  of  non-proprietary  OSA  and  standard  interfaces 

•  Facilitate  improved  systems  expansions  and  upgrades  in  response 
to  changing  missions  and  technology 

•  Major  Products 

-  Process  to  develop  Open  System  Architectures  tor  ships 

-  Open  CIC,  HVAC,  and  Environmental  Quality  Systems 
concepts  developed 

-  Technology  Management  for  DD21  and  LCS 

•  Architectures,  Interfaces,  and  Modular  Systems  (AIMS) 
current  ongoing  initiative  evolved  from  TOSA 
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ARCHITECTURES,  INTERFACES,  AND 
MODULAR  SYSTEMS  (AIMS)  PROGRAM 


•  Current  U.S.  Navy  RDT&E  Program  to  promote 
increased  Navy  use  of  OSA  and  modularity 

•  VISION 

-  To  create  a  Modular  Adaptable  Ship  (MAS) 
through  development  of  open  architecture  based 
zones  such  as  C4I,  Weapons,  and  Sensor  zones 

•  GOALS 

-  To  reduce  ship  life-cycle  costs 

-  Enable  technology  refresh  insertion 

-  Promote  competition 

-  Improve  mission  capability  and  flexibility 

-  To  facilitate  life-cycle  adaptability 

•  Examine  ship  designs  at  the  systems,  subsystems, 
and  component  level  to  determine  what  level  of 
modularity  makes  sense 

NDIA  Systems  Engineering  9 
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AIMS  VISION  -  MODULAR  ADAPTABLE  SHIP 


Mission  Mission 

Space  1  Space  2 


OPEN  FUNCTIONAL  ZONES 
^Modular  C4I  Zones 

•  Modular  Offboard  Vehicle  Zones 

•  Modular  Weapons  Zones 

•  Modular  Sensors  /  Topside  Zones 

•  Modular  Machinery  Zones 

•  Modular  Human  Support  Zones 

•  Other  (SOF  modules,  ISR,  modules) 


KEY  INTERFACES 

•  Data  &  information  (OACE) 

•  Physical  (Geometric  &  Tolerances) 

•  Weight  and  CG  /  VCG 

•  Services:  Electrical,  Air,Cooling 

•  Piping  connections 

•  Monitoring  &  Control  Sensors 

•  Human  Factors 

•  Survivability/Vulnerability:  shock, 
vibration,  EMI,  EMC,  etc. 


NAVY  AIMS  PROCESS 


REQUIREMENTS  ™tv  CDD’  sPecifications 


Key:  Technology 
Management-Continuous 
Market  Surveillance  & 
Technology  Projection 


Functional  Analysis 
And  Partitioning 

Select  Major  System  Architectures: 
•Adaptable  Ship,  Functional  Zones 
•Zonal  Distribution 
•Modular  Approach 


STANDARD 
INTERFACES 


ID  Options  & 
Functional  Interfaces 


VERIFICATION 


THE  ADAPTABLE 
SHIP 


Physical  Design 
Synthesis 
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CASE  STUDY  -  OSA  AND  MODULAR 
RECONFIGURABLE  SPACES 


User  Needs 

-Multi-Mission  Ship  on 
a  Single  Seaframe 

-Rapid  Mission 
Reconfiguration 

-Increase  Availability 

-Rapid  Technology 
Refresh  or  Insertion 

-Supportability 
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AIMS  Process  Execution 


Mission  Capable  Ship 
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REQUIREMENTS  ANALYSIS 


I  THE  ADAPTABLE 
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FUNCTIONAL  ANALYSIS 
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91919 


Weapons 

Systems 


Propulsion 

Systems 


HM&E 

Systems 


C4I 

Systems 


Mission 
System  1 


Mission 
System  3 


Mission 
System  2 


MODULAR  OSA 

HUMAN  SUPPORT  ZONE  TRADE  STUDY 


Configuration  Change 


Mission 

Storerooms 


Technological  Change 


Mission  Sensitivity 


r 

[SO  716 

6\ 

1 

Sulkhea 

d  ] 

Interfat 

:e  r 

Mission 

Workshop 


Market  Expansion 


Sanitation 

Vi  ■  ■  ■  V  Vi  tlvl  1 

Equipment 
Compatibility 


Reconfigurable 
*  Spaces  15 


FUNCTIONAL  AREA  SELECTION 

EXAMPLE 


Compartment  Attributes 

Attribute  Weighting 

5 

5 

5 

3 

2 

2 

2 

2 

5 

Estimated 
Number  of 
Compartments 
on  Ship 

Functional  Area 

Tech.  Change 

Configuration  Change 

Mission  Sensitivity 

Equipment  Applicability 

Storage  Efficiency 

Environmental  Change 

Sanitation 

Exposure  to  Elements 

Market  Expansion 

Ranking 

1  3-4  Mission 

11  Storerooms 

Mission  Storeroom:  Aviation  Storerooms, 
Hangers,  Workshops 

1 

5 

5 

5 

2 

3 

3 

5 

5 

121 

1 

Reconfigurable  Space 

2 

5 

5 

3 

1 

1 

3 

4 

5 

112 

1  11 

Stateroom  Crew  (4) _ 

1 

4 

1 

4 

1 

1 

4 

5 

5 

89 
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SEAFRAME 


OPEN  SYSTEMS  ARCHITECTURE 
MODULAR  RECONFIGURABLE  SPACE 


O  Key  Interfaces: 

Data  &  information  (OACE) 
Physical  (Geometric  & 
Tolerances) 

Weight  and  CG  /  VCG 
Services:  Electrical, 

Air,  Cooling 
Piping  connections 
Monitoring  &  Control  Sensors 
Human  Factors 
Survivability/Vulnerability: 
shock,  vibration,  EMI,  EMC, 
etc. 
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KEY  INTERFACES 

MODULAR  RECONFIGURABLE  SPACE 


Data 

Distributed  Systems  -  HVAC, 
electrical,  fluids,  etc. 

Structural  -  foundations 

-  International  Standards  Organization 
(ISO)  7166 

•  Aircraft  Rail  and  Stud  Configuration  for 
Passenger  Equipment  and  Cargo  Restraint 

•  Increase  core  modularity,  mission  readiness 
and  contain  costs  by  incorporating  ISO  7166 
bulkhead  interfaces 
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OPEN  SYSTEMS  ARCHITECTURE  AND 
STANDARD  INTERFACE  DEFINTIONS 


- UBlft.- 
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Reversible  Deck  Sockets 


Modular  Architecture 
Leverages  off  of  Previous  Programs 
ATC  Integrated  Joiner  Bulkhead  Project 
TOSA  Reconfigurable  Stowage 
Hull  Type  Drawings 


^ _ I 

3 - ^ 

tf 

BIN 

ISTORAGF 

"R" 

BIN 

storage: 

k  )  ks] 

pm  i  mm  j 

-DK. 


PLT. 


4h 
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ELEVATION 


Removable 

Perimeter 

Columns 


Overhead  Column 
Sockets 


Removable  Bulk 
Storage 
Telescoping  Battens 


MODULAR  OSA  SPACES  - 
RECONFIGURATION  OPTIONS 


MODULAR  OSA  KEY: 
INTERFACE  CONTROL 


Conceptual  Design 


I 


Sea  Frame  Development 

Preliminary  Design 


Final  Design 


High  Level  Impacts 

Refined  Architecture 

Interface  Specification  for 

to  Seaframe 

Required  for  Seaframe 

Detailed 

Design 

Architecture 

Development 

Interface  Control 

Perform  Concept 

Interface  Document 

Document  (ICD) 

Studies  to  Identify: 

•Mission  System  Physical  Requirements 

•Seaframe  definition: 

•Module  Stations  including 
Weapons,  Air,  Sea, 

Sensors,  and  Support 

•Detailed  foundation  definitions 

•Notional  Mission  Packages 

•Network 

•Communications 

•Baseline  Tech  Architecture  for  Mission 

•Command  and  Control  Software 

•Gross  Mission  Characteristics 

System  Interfaces: 

-Area,  Volume,  Weights 

•Mission  reconfiguration  definition: 

•Initial  Mission  Communications 

-Number  of  Module  Stations 

•Detailed  connection  definitions 

-Clearances 

•Focused  Mission  Package 

-Ship  Services:  Power,  Cooling, 
Air/Water,Data  Link 
-Launch,  Recovery  and  Handling 
-Core  and  Reconfigure  Systems 
-Stand  Alone  Resource  Stations 

-Ammunition 
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Ship  Life  Cycle  Phase 


Disposal 


Multi 

Bulk 


N/A  N/A 


Total 


85% 

I  40% 

1 90  Day 

|  80% 

270  Day 

ISO  7166  has  Slight  Decrease/  Increase  over  Conventional 
ISO  7166  has  Decrease/ Increase  over  Conventional 
ISO  7166  has  Significant  Decrease/Increase  over  Conventional 
ISO  7166  is  Equal  to  Conventional 


RECONFIGURABLE  SPACE  VERIFICATION: 
BUSINESS  CASE  ANALYSIS 
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Multi 

Bulk 


Development 


28  Day 
18  Day 


Procurement 


Multi 

Bulk 


-64% 

26% 


9% 

35% 


O&M/ 

Overhaul 


Multi 

Bulk 


90% 

70% 


1 62  Day 
252  Day 


28  Day 
18  Day 


BUSINESS  CASE  ANALYSIS 

RESULTS 


Conventional 

Layout  Compartment  (Chalk  locations) 

Subbases  or  Clips  (Foundation) 

Retrieve  From  Warehouse 

Scribed  for  local  irregularities,  sheer,  and 

camber 

Cut,  Weld  and  Watertight  sealed  to  the  deck 
Welding  "Flot  Work” 

Requires  Multiple  Fire  watches 

Working  and  Adjacent  Compartments  must  be 

prepped 

•  Removal  of  paint 

•  Removal  of  insulation 


Multipurpose  64  Days 
Bulk 


ISO  71 66 

Deck  Sockets  are  built  into  the  deck. 

Layout  Perimeter  (Chalk  Locations)  (accuracy  very  important) 
Deck  Clips  are  welded  around  the  perimeter  of  the  deck  on  a 
regular  grid  spacing. 

Overhead  sockets  are  welded  in  the  overhead  on  a  regular  gric 
spacing  interior  and  perimeter  ("Flot  Work”  Custom 
Manufacturing  and  Fitting) 

Underlayment  &  deck  covering  installed  (Where  specified) 
Perimeter  columns  inserted  in  overhead  socket  and  slid  over 
deck  clips,  bolted  in  place. 

Interior  Removable  Columns  are  installed  where  required 
Flip  deck  bolt  in  deck  socket 
Slide  column  in  overhead  socket 


New  Build 


52  Days 


Multipurpose  36  Days 


Bulk 


34  Days 


requiring  multiple  tool  sets,  techniques 
lust  be  installed 

Custom  Manufactured  to  fit  into  the  mountini 
location 

Weld  to  bulkhead  or  overhead  "Plot  Work” 
Doublers  must  be  installed  on  Stowage  Aid  in 
way  of  sway  brace  fasteners 
Clean  and  paint,  repair  insulation 


Conventional 


Conventional 

Engineering  Change  Drawings 
Stowage  Aid  Removal 

Empty  stowage  aid 

Remove  deck  covering  and  underlayment 
Unbolt  stowage  aid  from  sway  braces, 
subbase ,  clips  (foundation) 

Cut  sway  braces,  subbase  ,  clips  (foundation) 
from  ship  structure  ("Plot  Work’ ) 

Grind  smooth,  paint,  repair  insulation 
Layout  Compartment 
Subbases  or  Clips  (Foundation) 

Retrieve  From  Warehouse 

Scribed  for  local  irregularities,  sheer,  and 

camber 

Cut,  Weld  and  Watertight  sealed  to  the  deck 


ISO  7166 


ISO  7166 

Faster  and  Cheaper 


ISO  7166 

Interior  Removable  Columns  are  installed  or  removed  where  required 
Flip  deck  bolt  in  deck  socket 
Slide  column  in  overhead  socket 
Pin  column  through  deck  socket 
Stowage  Aid  Installation 

Retrieve  stowage  aid  from  warehouse 
Bolt  stowage  aids  to  ISO  track 


Reconfigurafion 


Multipurpose  59  Days 

Multipurpose 

5  Days 

Bulk  81  Days 

Bulk 

7  Days 

foundations;  repair  insulation 
Stowage  Aid  Installation  to  Foundation 

All  foundations  are  different 
Specific  Stowage  aid  that  mates  w 


Each  Stowage  Aid  mounts  in  a  different 
manner 

Sway  Braces  must  be  installed 

Custom  Manufactured  to  fit  into  the 
mounting  location 

Weld  to  bulkhead  or  overhead,  "Plot  Work” 

-  Doublers  must  be  installed  on  Stowage  Aid  ir 

way  of  the  sway  brace  fasteners 
Clean  and  paint,  repair  insulation 


ISO  7166 

Faster  and  Cheaper 


SUMMARY  AND  CONCLUSIONS 


•  Modular  reconfigurable  spaces 
based  on  OSA  and  Standard 
Interfaces: 

-  Cost  effective  solution  to  meet  User 
Needs. 

-  Satisfies  Capabilities  Requirements 
and  User  Needs  more  efficiently  and 
effectively  than  conventional  system. 

-  Enables: 

•  Mission  flexibility  (rapid  reconfiguration) 

•  Supportability  (common  components) 

•  Technology  refresh/insertion 
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OTHER  OSA  ACCOMPLISHMENTS  - 
INTERFACE  CONTROL  DOCUMENT  (ICD) 


•  Former  TOSA  team  members 
assigned  to  Mission  Systems  and 
Ship  Integration  Team  (MSSIT)  for 
a  major  ship  acquisition  program 

•  Developed  J-5  Appendix  to  RFP 
and  Contract:  ICD  Requirements 

-  Focused  initially  on  HM&E  interfaces 
for  preliminary  design 

-  Progressive  definition  to  include 
additional  interfaces 

•  Developed  J-l  0  Appendix  to  RFP 
and  Contract:  OSA  Open 
Architecture  Requirements 
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Abstract 

Threats  to  ground,  air,  space,  and  sea  are  increasingly  diverse,  and  the  large  set  of  operating  environments 
for  our  Armed  Forces  now  includes  non-traditional  roles  of  law  enforcement,  humanitarian  aid,  and 
homeland  defense  -  all  markedly  different  missions  that  must  be  performed  against  a  backdrop  of 
terrorism.  At  the  same  time,  long-time  threats,  such  as  cruise  and  ballistic  missiles,  are  becoming 
increasingly  sophisticated,  more  difficult  to  counter,  and  more  widely  distributed.  These  factors  pose 
critical  challenges  to  the  collective  defensive  posture  and  the  ability  to  achieve  fiscally  acceptable  solutions 
with  next  generation  systems.  After  reviewing  the  threats  and  problems  anticipated,  a  set  of  generic  key 
concepts  for  next  generation  combat  systems  (NGCS)  is  proposed:  aggregation,  automation,  and 
adaptation,  along  with  three  derivative  areas:  operational  control,  human  understanding,  and 
communications.  The  authors  propose  that  such  concepts  be  developed  and  built  into  systems  in  a  general 
and  consistent  manner. 

To  help  verify  these  key  concepts,  the  authors  illustrate  their  use  through  notional  application  to  the 
Ballistic  Missile  Defense  System  (BMDS).[1]  Finally,  the  relationship  between  these  key  concepts  and  the 
emerging  Global  Information  Grid  (GIG)  is  examined. 


[1]  Acronyms  are  also  expanded  on  the  last  page  of  text. 
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Combat  systems  have  evolved  considerably  over  time  to  reach  their  current  level  of  capability  and 
integration  among  component  systems.  In  early  systems,  “integration”  was  performed  by  people  using 
skills  and  processes  developed  through  training  and  experience  to  achieve  objectives  only  possible  through 
the  synergy  of  multiple  systems.  Key  in  the  evolution  of  the  Navy’s  current  generation  of  combatants  was 
the  development  of  the  Aegis  Combat  System.  In  Aegis,  the  Navy  achieved  an  integrated  fighting  ship  that 
“represented  a  major  transformation,  in  which  the  ship,  the  combat  systems  and  the  training  systems  were 
designed  as  a  single  unit.”[l]  Integration  among  systems  within  the  ship  became  the  realm  of  automation 
and  tailored,  managed  responses  governed  by  “doctrine”. 

Many  factors  are  pressing  the  evolution  of  combat  systems  beyond  our  current  legacy  capability. 
Achieving  the  “next  generation”  envisioned  on  the  right-hand  side  of  the  slide  will  require  us  to  address  a 
large  set  of  driving  functions  and  develop  technology  enablers. 

[1]  “History  of  Aegis,”  http://www.nps.navy.mil/meyerinstitute/history.htm. 
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Drivers 


•  Threat 

-  Innovation  in  tactics  &  weapons 

•  Complexity 

-  Unpredictability 

-  Situational  understanding 

-  Large  numbers 

-  Fragile  interdependence 

-  Denial  of  access  (spatial  &  system) 

-  Logistics 

-  Information  and  electronic  warfare 

•  Cost 

-  Operating  costs:  reduced  manning 

-  Life  cycle  costs:  common  components 

-  Refresh  cost:  shared  infrastructures 

-  Acquisition:  smaller,  more  specialized  units 


•  New  Environment 

•  New  Missions 

-  Diverse  Collaboration 

-  Homeland  defense 

•  Joint  &  Coalition 

-  GWOT 

•  Neutrals  and  civilians 

-  Global  in  spatial  extent 
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Drivers  for  Change 

While  the  current  state  of  combat  system  capability  is  unparalleled  in  naval  history,  complacency  is  not  acceptable. 
The  environment  in  which  naval  forces  operate  is  constantly  changing,  and  a  corresponding  evolution  of  our 
capabilities  is  required.  Drivers  for  this  evolution  include  the  following: 

Threat  -  The  weapons  employed  by  our  adversaries  continue  to  evolve.  Some  threats  are  evolving  in  technical 
sophistication,  making  it  increasingly  difficult  to  develop  single  combatant  solutions  that  provide  the  desired 
sureness  of  defense.  These  weapons  are  also  seeing  greater  proliferation. 

A  new  generation  of  cruise  missiles  . . .  could  lead  to  weapons  that  are  effective  to  ranges  of  approximately 
600-800  km.  While  primarily  pursued  as  naval  weapons,  conversion  to  land-attack  variants  would  be  a 
relatively  straightforward  process.  The  Iranian  anti-ship  cruise  missile  case  points  to  a  larger  problem  of 
attempting  to  control  the  spread  of  cruise  missiles:  an  increasing  number  of  suppli-ers,  low  cost,  ready 
availability  of  dual-use  technologies,  and  weak  international  controls.  In  acquiring  production  capabilities,  Iran 
is  also  poised  not  only  to  further  develop,  but  also  to  export  a  range  of  cruise  missiles[l] 

Enemy  states  and  organizations  are  also  evolving  unorthodox  approaches  that  stress  our  ability  to  adapt  systems 
and  methods  to  counter  them.  The  lessons  of  9/1 1  and  current  operations  in  Iraq  are  clear  evidence  of  a  trend  that 
enables  smaller  and  smaller  groups  to  have  significant  impact. 

Not  only  is  it  likely  that  many  of  the  conflicts  facing  the  West  will  be  of  an  asymmetrical  nature,  it  is  also 
likely  that  these  threats  will  come  from  diverse  and  simultaneous  sources.  For  example,  the  possibility  that 
conventional  terrorism  and  LIC  [low  intensity  conflict]  will  be  accompanied  or  compounded  by 
cyber/infrastructure  attacks,  damaging  vital  commercial,  military  and  government  information  and 
communications  systems,  is  of  great  concern.  In  this  sense,  a  major  Western  country  could  suffer  greatly  at  the 
hands  of  an  educated,  equipped  and  committed  group  of  fewer  than  50  people.  Such  an  attack  could  have  an 
effect  vastly  disproportionate  to  the  resources  expended  to  undertake  it[2] 

Under  other  potential  conflict  scenarios,  threats  may  appear  in  large  numbers,  stressing  our  ability  to  manage  a 
complete  response,  and  challenging  our  ability  to  create  cost-compatible  solutions. 

[1]  “Ra'ad  cruise  missile  boosts  Iran's  military  capability,”  Scott  Jones,  Jane’s  Intelligence  Review ,  1  April,  2004. 

[2]  “Intelligence  Gathering  on  Asymmetric  Threats  -  Part  One,”  Kevin  O’Brien  and  Joseph  Nusbaum  Jane’s  Intelligence 
Review ,  1  October,  2000. 
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New  Missions  -  The  mission  set  for  Navy  combat  systems  is  constantly  expanding.  Humanitarian  aid,  low 
intensity  conflict,  law  enforcement,  and  terrorist  attacks  were  not  primary  concerns  (or  even  envisioned  in  some 
cases)  when  constructing  many  of  today’s  combat  systems.  New  mission  needs  are  pushing  the  Navy  out  of  its 
traditional  operating  format.  GWOT  requires  the  ability  to  respond  rapidly  and  simultaneously  in  many  areas  of 
the  world.  As  described  in  a  draft  Navy  strategy[l],  four  national  security  challenges  stem  from  the  GWOT: 
irregular  (uncon-ventional  methods),  catastrophic  (rogue  employment  of  weapons  of  mass  destruction  [WMD]), 
traditional,  and  disruptive  (application  of  breakthrough  technologies). 

“The  agility  of  operational  deployed  naval  forces  supporting  a  Joint  Force  Commander  provide  the 
United  States  with  extraordinary  overseas  reach. .  .The  increasingly  urgent  task  for  the  Navy,  and  the 
larger  Joint  Force,  is  to  determine  what  forces  and  concepts  are  required  to  meet  the  four  challenges 
outlined  by  the  Secretary  of  Defense”[l]. 

The  difficult  part  is  that  these  will  be  sustained  mission  obligations  with  three  of  the  four  demanding  an 
innovation  cycle  much  shorter  than  for  traditional  combat  equipment.  Combat  system  responses  must  vary 
dramatically  depending  on  these  missions,  and  future  combat  systems  must  be  able  to  configure  themselves 
rapidly  and  easily  to  the  future  changing  mission  environment. 

New  Operating  Environment  -  Joint  operation  is  expected  to  predominate,  and  coalition  operation  will  become 
even  more  common.  “The  joint  force,  because  of  its  flexibility  and  responsiveness,  will  remain  the  key  to 
operational  success  in  the  future.”[2]  While  this  is  not  a  new  trend,  the  extent  to  which  military  planners  are 
incorporating  it  as  standard  procedure  is  of  note.  There  is  still  much  to  do  in  aligning  our  individual  combatant 
capabilities  to  the  notion  of  a  coherently  operated  joint  force.  In  an  address  to  the  17th  International  Seapower 
Symposium,  Admiral  Mike  Mullen,  Chief  of  Naval  Operations,  extended  this  principle  to  encompass  the 
international  Navy  community  in  commenting  on  the  difficulty  of  addressing  “irregular  and  unrestricted 
warfare”[3]:  “Perhaps  the  most  profound  effect  of  today’s  challenges  is  the  increased  value  of  cooperation 
between  friends,  allies,  coalition  partners,  and  like-minded  nations.  Despite  differences  in  size  or  structure  of  our 
navies,  cooperation  today  is  more  necessary  than  ever  before”. 

Complexity  Management  -  As  combat  systems  have  evolved,  they  have  become  more  complex,  making  the  task 
of  effective  employment  increasingly  difficult.  These  complexities  must  be  explicitly  recognized  and  addressed 
in  future  combat  systems.  A  key  example  is  the  difficulty  we  have  in  understanding  the  tactical  situation:  what  are 
we  in  a  position  to  do,  what  will  our  automation  do  without  our  intervention,  and  what  new  courses  of  action 
should  be  formulated.  Increasing  the  interdependency  among  components  of  a  system-of-systems  introduces 
additional  complexity.  Widespread  joint  operations  and  their  associated  component  interdependencies  pose  that 
risk.  Efforts  to  address  robustness  and  integrity  must  keep  pace  with  the  growth  toward  more  interconnected 
capabilities  to  stem  the  tendency  toward  fragility.  While  often  viewed  as  mundane,  the  ability  to  support 
operations  must  also  be  considered.  As  stated  recently,  “Absent  a  concomitant  revolution  in  the  support  activities 
of  defense,  the  Revolution  in  Military  Affairs  will  quickly  outrun  the  ability  of  logistics,  personnel,  medical  and 
other  systems  to  support  it.”[4] 

Cost  -  Pressure  to  control  cost  is  present  in  all  aspects  of  new  and  existing  systems:  development,  production, 
operation,  and  maintenance.  Achieving  standardization  of  function  and  interfaces  is  an  approach  to  more 
efficiently  developing  elements  of  the  combat  system.  It  allows  them  to  be  used  across  many  systems  which  will 
share  the  development  and  maintenance  burden.  The  Navy  is  exploring  the  concept  of  smaller,  more  mission- 
focused  combatants  (e.g.  littoral  combat  ship).  While  the  smaller,  more  focused  unit  provides  a  lower  unit 
production  cost,  it  will  need  to  work  closely  together  and  with  other  joint  assets  in  order  to  fulfill  all  missions. 
Reduced  manning  on  the  Navy’s  major  combatants  has  been  a  cost  reduction  objective  for  quite  some  time. 
Progress  in  the  operator/decision  maker  area  has  improved  the  ability  to  accomplish  more,  but  it  is  unclear 
whether  the  current  pace  of  progress  is  keeping  up  with  the  growing  complexity  of  the  systems,  number  of 
options,  and  collection  of  roles  that  face  the  modern-day  warfighter. 

[1]  “Navy  3/1  Strategy:  The  Maritime  Contribution  to  the  Joint  Force  in  a  changed  Strategic  Landscape,”  N3/N5,  April,  2005. 

[2]  “Joint  Vision  2020,”  Chairman  of  the  Joint  Chiefs  of  Staff,  May  2000. 

[3]  From  “Remarsk  as  Delivered  for  the  17th  International  Seapower  Symposium,”  ADM  Mike  Mullen,  Naval  War  College, 

21  September,  2005,  as  recorded  in  http://www.chinfo.navy.mil/navpalib/cno/speeches/mullen050921.txt. 

[4]  From  leading  change  in  a  new  era  http://www.defenselink.mil/pubs/dodreform/fullreport.pdf 
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Key  Concepts  for  NGCS 


Drivers 

Threat 
Complexity 
New  Missions 
New  Environment 
Cost 


Values 

Effective  use  of  Joint  assets' 
Flexibility  in  response 
Coordination/Cooperation  of 
Joint  Forces 

Broad,  detailed  situational 
understanding 
Fast,  accurate  response 
Information  assurance  and 
pedigree 
Simplicity 


Primary  Concepts 

-  Aggregation 

-  Decision  Automation 

-  Dynamic  Adaptation 


Derived  Concepts 

-  Operational  Control 

-  Enhanced  Awareness  & 
Understanding 

-  Communications  infrastructure 
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Concepts 

There  are  many  viewpoints  painting  a  vision  of  the  operational  future,  the  needs  of  the  various  services  in 
the  coming  era,  and  the  transformational  efforts  that  will  be  required  to  reach  these  goals.  Some  of  these 
visions  are  documented  in  references  cited  throughout  this  paper.  The  fusion  of  these  many  thoughts  into  a 
small  number  of  key  concepts  was  a  difficult  exercise  in  synthesis  that  applied  loosely  structured  events 
and  exercises.  Ideas  on  potential  concepts  and  their  relationships  to  drivers  and  values  were  collected  and 
merged  where  possible.  We  made  significant  use  of  “mindmaps”  [1]  to  organize  ideas  from  reference 
sources,  and  later,  to  consolidate  them.  Governing  this  effort  was  a  theme:  What  fundamental  concepts 
could  be  developed  or  extended  to  enhance  the  capabilities  of  the  combat  system  no  matter  what  the 
mission,  no  matter  what  new  weapon  technology  might  bring,  and  no  matter  what  tactics  an  adversary 
might  apply?  The  notion  is  that  key  advances  in  integration  of  capabilities  (at  the  combat  system  level)  if 
established  in  a  common  form,  can  amplify  the  steady  march  of  progress  in  sensors,  and  weapons  - 
independent  of  specific  technology. 

The  concepts  that  emerged  from  this  effort  appear  in  the  slide  above:  aggregation,  automation,  and 
adaptation  are  three  primary  concepts[2],  with  operational  control,  enhanced  awareness  and  understanding, 
and  communications  infrastructure  as  “derived”  concepts.  (The  derived  concepts  support  the  primary 
concepts.)  Each  of  these  is  defined  below,  with  the  primary  concepts  addressed  in  more  detail  in 
subsequent  sections.  (Amplification  of  the  derivative  concepts  will  be  reserved  for  a  later  paper.) 

Instances  of  these  concepts  have  emerged  with  some  of  our  more  advanced  capabilities.  However,  they 
have  emerged  in  specialized  form  for  particular  domains.  It  is  felt  that  these  concepts  are  (or  should 
become)  fundamental  tools  in  modern  combat  systems.  They  should  be  built  into  the  system  at  the  most 
fundamental  levels  and  in  a  generic  way.  This  will  allow  the  concepts  to  permeate  the  combat  systems  of  a 
force  and  bring  about  a  dramatic  magnification  of  capabilities. 

[1]  “Definition  of  Mindmaps,”  Tony  Buzan,  http://www.mind-map.com/EN/mindmaps/definition.html 

[2]  It  is  important  to  note  that  we  still  consider  this  a  work  in  progress.  We  have  a  strong  feeling  that  there 
may  be  more  “primary”  concepts  that  we  have  not  yet  labeled. 
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Aggregation  is  the  pooling  of  resources  from  independent  units  to  collaboratively  perform  mission  tasks.  In 
aggregation,  resources  (or  portions  of  them)  from  independent  systems  are  nominated  to  be  constituents  of 
a  resource  pool.  Those  resources  are  then  applied  to  broad  mission  objectives  that  might  be  unachievable 
by  any  individual  unit.  How  such  resources  are  identified,  partitioned,  tasked,  and  controlled  is  critical  to  a 
robust  operational  capability.  These  form  the  primary  areas  of  interest  for  aggregation  concept 
development. 

(Decision)  Automation  is  the  ability  of  a  “system”  to  autonomously  initiate  actions  or  develop  alternatives 
for  human  decision.  This  is  not  a  new  concept;  it  is  quite  analogous  to  the  “automated  doctrine”  used  in 
Aegis.  The  difference  is  in  the  breadth  of  generality  and  capability  that  we  are  striving  for.  The  goal  is  to 
significantly  increase  the  set  of  information  on  which  decisions  are  based  (including  expanding  that 
information  set  beyond  the  confines  of  the  individual  unit)  and  to  grow  the  collection  of  “actions”  that  can 
be  taken.  The  term  “actions”  is  intended  to  be  very  encompassing,  e.g.,  it  includes  the  capability  to  prompt 
the  human  decisionmaker  to  action,  issue  alerts,  provide  recommendations,  alter/highlight  displays,  or  even 
modify  the  internal  processing  (parameters  or  rules)  of  a  system  component.  Decisions  are  automated 
through  a  rich  set  of  rules  linking  the  initiation  of  each  action  to  specific  observable  events. 

Adaptation  is  the  ability  of  the  combat  system  to  respond  rapidly  to  changes  in  asset  participation, 
environment,  mission,  or  threat.  Our  forces  operate  in  an  environment  in  which  change  is  constant.  Threat 
tactics  change,  the  environment  changes,  systems  arrive  and  depart  as  participants  in  a  force,  and  the 
capabilities  of  those  systems  change  with  upgrades  and  new  installations.  This  requires  us  to  establish  a 
mature  approach  to  managing  responses  to  the  various  types  of  change.  Change  cannot  be  considered  an 
anomaly. 

Operational  Control  is  the  ability  to  monitor  and  manage  the  dynamic  automated  aggregate  operations. 
There  are  two  very  different  aspects  to  this  concept.  First,  the  appropriate  interactions  among  decision 
makers  at  all  levels  of  command  must  be  established  and  supported.  Passing  of  “control”  (or  even  lesser 
responsibility)  of  a  resource  from  the  unit  that  owns  it  to  another  unit  that  may  be  making  more  global 
decisions,  is  a  significant  change  that  must  be  considered  by  the  operational  community.  Establishing  the 
principles  by  which  that  may  occur  (in  a  general  sense)  will  be  a  significant  undertaking  but  is  critical  to 
advancing  the  aggregation  concept.  Secondly,  there  must  be  reliable  and  predictable  control  of  the 
aggregate  system  (including  its  automation  and  adaptation  functions)  in  order  for  these  concepts  to  be 
operationally  acceptable.  More  rigorous  approaches  to  ensuring  that  components  will  behave  as  desired, 
and  only  as  desired,  must  be  established. 

Enhanced  Awareness  and  Understanding  is  the  ability  to  develop  comprehensive  situational 
understanding.  This  is  a  long  pursued  objective,  and  progress  has  been  made  in  many  existing  and 
emerging  systems.  But  it  is  also  clear  from  observations  of  the  Human  Machine  Interface  aboard  the  USS 
Ronald  Reagan  that  there  is  yet  work  to  do  [1].  The  growth  of  aggregate  functionality  and  increased 
automation  will  also  increase  the  complexity  of  understanding  how  systems  will  respond  and  what  controls 
need  to  be  manually  asserted.  Simply  understanding  “what  will  the  system  do  if  I  leave  it  alone”  is  not  a 
trivial  exercise. 

Communications  infrastructure  comprises  the  services  that  enable  collaboration  among  aggregate 
components  and  warfighters.  None  of  this  will  happen  without  communications  and,  more  importantly, 
without  communications  that  is  much  more  capable,  predictable,  and  robust  than  currently  available.  The 
establishment  of  the  GIG  recognizes  this  need[2].  The  key  is  for  the  GIG  development  to  fully  recognize 
the  communication  requirements  of  combat  systems. 

[1]  “Sail- Around  Evaluation:  The  Battle  Management  Organization  and  Human  Machine  Interface  as  part 
of  the  USS  Ronald  Reagan  Combat  System,”  Draft  Report,  Technology  Management  Group,  Inc., 

February  2005. 

[2]  Global  Information  Grid  Capstone  Requirements  Document,  JROCM  134-01,  30  August,  2001 
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Aggregation 

Aggregation  is  simply  the  application  of  all  available  and  appropriate  resources  to  a  mission,  independent 
of  which  unit  hosts  them.  Examples  of  aggregation  are  emerging  in  current  systems.  The  Ballistic  Missile 
Defense  System’s  (BMDS)  overall  concept  is  clearly  one  in  which  a  diverse  collection  of  assets  is  applied 
to  multi-mission  objectives  of  national  missile  defense  and  regional  defense  against  short  and  long  range 
ballistic  missiles.  No  single  system  or  unit  can  “solve”  the  problem.  It  is  only  through  the  synergy  of 
multiple  sensor  and  weapon  systems  (and  their  associated  combat  systems)  that  operationally  viable 
solutions  are  achieved.  The  slide  above  depicts  a  notional  interconnect  of  different  components  in  the 
BMD  system.  The  Command  and  Control  Battle  Management  (C2BM)  component  of  that  enterprise  is 
tasked  with  providing  the  required  coordination  among  components  to  support  the  complex  objectives.  A 
second  example  of  emerging  aggregation  is  the  Navy’s  Cooperative  Engagement  Capability  (CEC).  This 
provides  a  sensor  information  sharing  and  integration  system  that  creates  improved  tactical  awareness  and 
also  enables  multi-unit  supported  guidance  for  engagements!!].  It  seems  that  we  are  on  the  front  edge  of  a 
technologically  supported  ability  to  reap  the  benefits  of  much  more  closely  coordinated  behavior  among 
our  individual  systems.  As  suggested  by  the  Undersecretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  at  an  Armed  Forces  Communication  and  Electronics  Association  (AFCEA)  conference,  “I  can 
think  of  no  more  critical  need  than  the  development  and  fielding  of  a  joint  battle  management  capability. 

...  A  key  objective  is  to  provide  robust  capabilities  and  innovative  approaches  for  the  full  spectrum  of 
potential  missions  using  a  system  of  systems  approach.”[2] 

[1]  “CEC:  Sensor  Netting  with  Integrated  Fire  Control,”  C.  J.  Grant,  Johns  Hopkins  APL  Technical  Digest 

Vol  23,  Nos.  2  and  3, 2002.  "  " 

[2]  As  reported  in  “CHIPS  -  The  Department  of  the  Navy  Information  Technology  Magazine,”  Summer 
2004,  http://www.chips.navy.mil/archives/04_summer/Web_pages/Michael_Wynne.htm. 
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Spectrum  of  Aggregate  Combat  Systems 


•  Dynamic  pool  of  assets 

•  Cross  mission  real-time  optimization  of  resources 

•  Real-time  automated  collaborative  C2 


•Selected  resources  “yield”  control  to 
distributed  command  pool 
•  Limited  automated  optimization  support  within 
single  mission  areas 


Spectrum  of  Aggregation 


•  Pool  of  some  resources 
>  Local  decision  optimization 


•  Fixed  set  of  interchangeable  resources 

•  Manual  decision  process 

*  Resources  “owned”  by  individual  units/commanders 

*  Coordination  through  planning  and  commands 


Significant  “dimensions”: 

•  Dynamics  of  participation 

•  Application  /  breadth  of  optimization 

•  Portion  of  assets  included  (sensors, 

actors,  decision-makers) 
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We  can  look  at  aggregation  as  a  spectrum  of  behavior  (depicted  above)  ranging  from  simple  single¬ 
objective  human  coordinated  efforts  to  fully  automated,  real-time  optimization  of  resource  use  across 
multiple  missions.  While  drawn  as  a  single-dimension  continuum,  the  spectrum  has  somewhat  independent 
dimensions  of  breadth  (the  number  of  resources  addressed),  scope  (the  set  of  objectives  simultaneously 
addressed),  and  dynamism  (the  ease/speed  with  which  the  management  adapts  to  new  conditions).  The 
appropriate  working  point  within  this  spectrum  depends  on  the  state  of  technology  to  support  it.  The 
technology  task  ahead  is  to  push  the  operating  point  forward;  the  systems  engineering  job  is  to  select  the 
appropriate  operating  point  for  any  given  time. 
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Throughout  the  paper,  we  examine  the  application  of  these  concepts  to  the  BMD  domain.  In  this  extremely 
challenging  technical  problem,  we  find  multiple  sensors  of  varying  types  that  can  be  applied  to  the  detection  and 
tracking  problem,  multiple  weapons  that  have  varying  ranges,  and  hosts  that  vary  in  mobility  and  command  structure. 
The  slide  above  depicts  a  notional  case  in  which  a  land-based  sensor,  two  ship  sensors,  and  a  satellite  are  available  to 
support  detection  and  tracking  of  ballistic  missile  launches  in  the  region.  To  help  raise  the  likelihood  of  detection,  the 
land-  and  sea-based  sensors  focus  their  detection  energy  on  areas  determined  by  combinations  of  launch  and  impact 
points.  The  satellite  has  a  broader  field  of  view  for  detection,  but  lacks  precision  for  tracking  and  may  suffer  from 
time- varying  characteristics:  It  may  not  always  be  in  place  and  does  not  always  have  a  clear  view  (due  to  atmospheric 
interference).  The  problem,  very  simply,  is  to  use  these  assets  to  the  best  of  their  abilities,  in  combination,  to  provide 
the  most  reliable  and  accurate  detection  and  tracking  of  ballistic  missiles  possible  (with  those  assets).  The  changing 
nature  of  the  environment,  intelligence  (anticipated  launch  points),  the  resources  themselves  (satellite  and  ship 
locations),  etc.,  establish  a  dynamic  environment  in  which  this  optimization  is  performed. 

To  support  the  aggregation  concept  in  a  general  way,  capabilities  must  be  established  that  allow  resources  from 
independent  units  to  be  grouped  for  optimized  application  to  a  mission.  Mechanisms  must  be  established  to  do  the 
following: 

•Create  a  pool  of  resources  (in  this  case,  the  sensors). 

•Nominate  resources  to  the  pool,  i.e.,  give  approval  for  the  nominated  resource  to  be  used  (as  determined  by  the 
resource  manager).  It  also  seems  likely  that  a  mechanism  for  (optional)  final  approval  be  established  to  allow  the 
resource  owner  final  say  on  how  a  resource  might  be  used. 

•Express  a  mission  objective  in  a  way  that  allows  a  resource  manager  to  discover  a  “good”  (ideally  optimal) 
resource  allocation  to  fulfill  it.  Mechanisms  for  feedback  to  human  decisionmakers  and  thresholds  for 
acceptability  are  needed  to  provide  the  resource  manager  the  needed  guidance  on  when  “best  effort”  is  suitable 
and  when  decisionmakers  must  enter  the  picture  to  consider  the  problem  (with  additional  options  not  available  to 
the  resource  manager). 

•Provide  a  mechanism  for  managing  the  resources  in  a  pool,  allocating  them  to  specific  responsibilities  that 
collectively  fulfill  a  mission  objective.  (This  includes  the  “optimization”  function  that  here  must  provide  best- 
effort  solutions  to  prioritized  objectives  under  any  collection  of  resources  and  objectives.) 
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Decision  Automation 
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Automation 

Automation  is  a  broad  term.  In  Aegis  as  well  as  in  many  other  combat  systems,  a  set  of  automated 
capabilities  already  exists,  including  automated  detection  and  tracking,  automated  status  reporting,  and 
readiness  assessments.  The  focus  here  is  decision  automation,  which  also  has  an  existing  set  of  capabilities 
supporting  automated  and  human- supported  decisions  involved  in  object  identification,  threat  engagement, 
and  radar  control.  Automation  joins  our  list  of  key  concepts  due  to  its  continuing  importance. 

The  current  operational  context  includes  large  collections  of  sensors,  weapons,  systems,  rules  for  use, 
preferred  application  techniques,  etc.  While  these  provide  capability  (and  options),  they  substantially 
complicate  decisions  about  what  to  do  and  when.  The  intended  extension  of  automation  we  seek  is  the 
significant  broadening  of  all  aspects  of  the  capability:  the  input  set  that  is  applied  to  the  decision  process,  the 
richness  of  the  language  that  is  used  to  express  desired  automated  actions,  and  the  set  of  actions  that  may  be 
taken  as  a  result  of  automation.  Moreover,  it  is  desired  that  these  extensions  take  a  form  that  can  be  applied 
in  a  regular  fashion  to  all  the  component  systems  (and  their  associated  controls)  that  comprise  the  combat 
system  “system  of  systems.” 

In  the  development  of  Aegis,  the  simple  model  “detect,  control,  engage”  was  used  to  provide  a  high  level 
characterization  of  its  overall  architecture  and  automation  approach.  In  considering  the  growth  of  automation 
for  the  future  combat  system  context,  a  more  apropos  model  is  “sense,  decide,  act,”  where  the  growth  in 
domain  of  all  three  steps  is  generalized  to  the  broader  mission  context.  In  this  model,  automation  and  human 
decision  must  collaborate  effectively  and  efficiently  to  address  the  staggering  number  of  options  that  are 
afforded  by  increasingly  capable  and  flexible  systems.  In  its  simplest  form,  sensed  data  and  information  are 
used  to  decide  when  and  under  what  conditions  various  system  capable  actions  are  initiated. 

An  important  principle  in  these  systems,  however,  must  continue  to  be  the  ability  of  the  human 
decisionmakers  to  exercise  supervisory  control.  This  really  requires  two  things”:  first,  that  appropriate 
control  mechanisms  exist,  and  second,  that  the  behavior  of  the  system  be  comprehensible  to  the 
decisionmaker.  As  more  automation  is  established,  both  of  these  become  more  difficult  to  achieve  and  new 
techniques  may  well  be  required. 
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Automation  in  a  ballistic  missile  defense  scenario  should  assist  in  many  of  the  operational  decisions  that  may 
occur.  The  slide  above  depicts  a  complex  situation  involving  multiple  ballistic  missiles,  multiple  sensors, 
multiple  defensive  firing  units,  and  multiple  defended  areas.  It  is  annotated  with  some  of  the  decision  points 
listed  below,  where  we  would  expect  that  broad  automation  capabilities  would  play  a  major  role  in  collecting 
relevant  information  and  either  making  decisions  outright  or  providing  recommendations  or  option  summaries 
for  human  decision. 

1.  For  any  initial  detection  (especially  one  that  is  the  “first”  in  an  actual  military  exchange),  there  are 
significant  decisions  that  must  be  made  on  whether  the  observation  is  indeed  a  real  object  and  whether  it  is 
one  of  interest,  e.g.,  a  ballistic  missile  of  some  sort. 

2.  Given  that  a  real  missile  has  been  detected,  the  next  set  of  questions  pertains  to  whether  it  is  something  that 
warrants  engagement.  Is  it  perhaps  just  a  test?  If  it  is  indeed  a  hostile  action,  is  it  of  sufficient  concern  that 
we  should  attempt  engagement?  (An  answer  to  this  question  varies  considerably  depending  on  prior  events 
and  weapon  stores.) 

3.  On  the  event  of  a  verified  first  hostile  launch,  there  may  be  many  actions  (changes  to  automation  settings, 
for  example)  that  might  need  to  be  altered  to  establish  a  more  active,  faster  response  to  further  hostilities. 

4.  A  ballistic  missile  in  flight  may  be  trackable  by  multiple  assets.  Which  ones  should  be  used?  Are  there 
sensor  resource  loading  issues  to  address  if  there  are  multiple  missiles  in  flight?  Are  different  sensors  better 
equipped  to  address  different  phases  of  tracking  and  discrimination?  (Clearly,  this  ties  into  the  aggregation 
concept  quite  directly.) 

5.  For  a  missile  that  is  to  be  engaged,  selection  of  the  engaging  unit  and  weapon  must  be  made.  What  strategy 
should  be  applied  to  the  engagement:  salvo,  shoot-look-shoot,  single-shot? 

6.  For  an  engaged  target  complex,  an  assessment  must  be  done  to  determine  whether  the  warhead  has  been 
neutralized.  This  may  also  lead  to  a  decision  on  whether  to  reengage,  with  criteria  for  reengagement 
changing  as  a  result  of  the  tactical  situation  and  number  of  remaining  defensive  missiles.  Again,  a  choice  of 
engaging  unit  and  weapon  must  be  made  (when  more  than  one  option  remains). 
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Adaptation 


Adaptation:  The  ability  of  the  combat  system  to  respond  rapidly  to 
changes  in  asset  participation,  environment,  mission,  or  threat. 


Aspect 

•  Focuses  on  the  infrastructure  required  to  adapt  to  or  to  institute 
change  at  various  levels  of  the  aggregated  combat  systems. 

-  Asset  participation:  protocol  for  adjusting  to  the  entry/exit  of  a  unit  or 
its  resources  from  participation  in  one  or  more  aggregate  resource 
pools 

-  Operational  doctrine  modification:  ability  for  unit  personnel  to 
adjust  or  select  automation  rules  in  response  to  an  operational 
situation 

-  Engineering  doctrine  modification:  ability  to  adjust  combat  system 
parameters  and  automation  rules  and  forward  them  for  ship  use 
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Adaptation 

A  century  ago,  German  strategist  Field  Marshal  Helmuth  von  Molke  warned,  “No  plan  survives  the  first 
engagement  with  the  enemy’s  main  force.”£lJ  The  creativity  and  innovation  shown  by  current  enemies  seem  to 
bear  this  out  as  a  continuing  principle.  The  risk  in  creating  more  complex  collaborative  and  automated 
approaches  to  warfare  and  defense  is  that  we  may  create  capabilities  that  are  overly  rigid  and  vulnerable  to 
unanticipated  innovation. 

Our  focus  on  adaptation  concepts  is  pointedly  aimed  at  avoiding  that  problem.  We  envision  three  types  of 
adaptation  (as  shown  above): 

•  Changes  in  resources/participation  -  previously  discussed  as  an  element  of  aggregation  (and  indeed  it  is)  in 
which  the  overall  force  must  adapt  in  real  time  to  the  presence  of  different  resource  sets  under  a  continuing 
fixed  set  of  mission  objectives.  A  broad  interpretation  of  this  also  encompasses  the  evolution  of  systems: 
one  may  see  an  airborne  warning  and  control  system  (AWACS)  aircraft  depart  and  a  new  one  arrive  on 
station  with  significantly  different  capabilities. 

•  Field  modification  of  automation  -  With  suitable  support  for  entering  and  validating  new  automation  rules, 
we  wish  to  enable  the  warfighter  community  to  tailor  automated  responses  to  the  demands  of  their  particular 
environment  and  operational  guidelines.  Achieving  this  requires  a  balance  between  flexibility  and 
complexity,  hopefully  elevated  by  effective  human  interface  design. 

•  Engineering  station  modification  of  automation  -  There  are  elements  of  automation  control  that  may  be 
beyond  the  expected  proficiency  of  the  warfighter.  These  areas  can  still  be  addressed  as  adaptable  functions 
but  with  the  support  of  an  engineering  community  at  a  base  or  command  installation.  In  addition  to 
personnel  with  additional  training,  such  a  site  might  also  have  considerably  greater  assets  for  the 
determination  of  optimal  automation  settings  and  for  the  validation  of  what  might  be  a  large  set  of 
interrelated  adjustments. 

[11  cited  in  “Transforming  military  improvisation  into  strategy,”  The  Lawton  Constitution.com,  Richard  Hart 
Sinnreich,  http://www.lawton-constitution.com/sinnreich/archives/ 
Transforming%20military%20improvisation%20into%20a%20strategy.htm. 
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Warfighter 


Adaptation  in  BMD  Scenario 

Engineering  Center 


Adaptation  capitalizes  on  the  assets  of  the  deployed  fleet  and  its 
engineering  support  centers  to  rapidly  react  to  changes  in  the  threat 
and  enemy  tactics. 
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Adaptation  in  a  ballistic  missile  defense  scenario  might  follow  many  paths.  First,  as  discussed  initially 
under  aggregation,  the  set  of  resources  to  be  applied  to  the  detection  and  tracking  problem  should  be  seen 
as  a  set  under  the  constant  potential  of  change  due  to  environment,  system  availability,  and  even  pressing 
needs  in  another  mission  area.  Second,  on  the  event  of  the  first  confirmed  tactical  ballistic  missile  (TBM) 
launch,  it  is  very  likely  that  many  identification  (ID),  tracking,  and  engagement  controls  might  be  altered  to 
operate  more  aggressively  (and  with  less  command  level  confirmation)  to  allow  weapons  and  systems  to 
be  used  to  their  greatest  effectiveness.  Third,  the  experience  of  engaging  the  enemy  might,  for  example, 
reveal  an  unexpected  decoy  approach.  With  the  support  of  remote  engineering  analysis  fueled  by  detailed 
field  sensor  data,  the  algorithm  for  identifying  the  ballistic  missile  warhead  might  be  adapted  to  yield  a 
lower  susceptibility  to  deception.  A  network-based  delivery  of  the  new  parameters  for  identification 
automation  would  be  provided  and  installed  with  as  quick  a  turn  around  as  possible. 
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GIG  Definition  and  Objectives 


•  Definition: 

Global  Information  Grid:  “The  globally  interconnected,  end-to-end  set  of  information 
capabilities,  associated  processes,  and  personnel  for  collecting,  processing,  storing, 
disseminating  and  managing  information  on  demand  to  warfighters,  policy  makers, 
and  support  personnel.” 

•  Needs 

-  “The  GIG  shall  support  all  DoD  missions  with  information  technology...” 

-  “The  GIG  assets  shall  be  interoperable  ...” 

-  “The  GIG  shall  be  based  on  a  common,  or  enterprise  level,  communications  and 
computing  architecture  to  provide  a  full  range  of  information  services  at  all  major 
security  classifications...” 

•  Vision 

“U.S.  forces  must  leverage  information  technology  and  innovative  network-centric 
concepts  of  operations  to  develop  increasingly  capable  joint  forces.  Our  ability  to 
leverage  the  power  of  information  and  networks  will  be  key  to  our  success  in  the 
21st  century.  We  must  make  information  available  on  a  network  that  people  will  be 
willing  to  depend  on  and  trust.  We  must  populate  that  network  with  new  types  of 
information  needed  to  defeat  future  enemies  and  make  existing  information  more 
readily  available.  And  we  must  deny  enemies’  information  advantages  against  us.” 
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Key  Concepts  and  the  GIG 

Policy  for  the  GIG[1]  aims  for  a  future  in  which  virtually  all  information  systems  (and  those  that  employ  or 
provide  information  to  them)  are  crafted  to  operate  in  a  common  information  exchange  environment.  With 
standards  guiding  interoperability,  enterprise-level  services  providing  common  and  efficient  base 
capability,  and  a  greatly  extended  communications  infrastructure,  the  GIG  promises  to  be 
“transformational”  in  the  truest  sense  of  the  word.  It  is  critical  then  to  consider  how  this  much-elevated 
capability  for  delivering  and  processing  information  may  influence  the  combat  systems  of  the  future. 

The  Navy  has  established  a  specific  initiative  for  pursuit  of  GIG  objectives,  called  FORCEnet,  which 
specializes  the  requirements  of  the  GIG  to  the  Navy  environment  and  provides  a  first-level  view  of  a 
planned  architecture [2].  Interestingly,  the  future  combat  system  concepts  we  espouse  most  certainly  fit 
under  the  GIG  umbrella  of  a  system  that  “transmits  information  to,  receives  information  from,  routes 
information  among,  or  interchanges  information  among  other  equipment,  software,  and  services.”[l] 
However,  historic  differences  in  requirements  and  the  ability  of  technology  to  meet  them  have  fueled  two 
independent  development  communities:  the  combat  system  and  information  system  (sometimes  known  as 
command,  control,  communications,  computers,  and  intelligence  [C4I]  communities). 

The  convergence  of  these  two  futures  seems  inevitable.  The  true  “value”  of  the  the  GIG  and  its  Navy 
FORCEnet  manifestation  is  not  so  much  its  ever  more  powerful  knowledge  base  but,  rather,  what  can  be 
done  with  the  knowledge  it  creates.  A  favorite  term  in  the  intelligence  community  (which  is  a  major  player 
in  the  information  system  environment  of  the  GIG)  is  “actionable  intelligence.”  The  future  combat  systems 
will  be  primary  providers  of  that  “action.” 

[1]  “Global  Information  Grid  (GIG)  Overarching  Policy,”  DoD  Directive  8100.1,  Deputy  Secretary  of 
Defense  Paul  Wolfowitz,  19  September,  2002. 

[2]  ’’FORCEnet  Architecture  Vision,”  Version  1.2,  Office  of  the  SPAWAR  Chief  Engineer  SPAWAR  05, 
18  July  2003. 
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Next  Generation  Combat  System 
Concepts  and  the  GIG 


Information  and  Knowledge  Gathering  and  Processing 
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Interacting  Combat  Systems 


Combat  systems 
participating  in  an 
aggregate  to  share  mutually 
beneficial  resources  utilize 
the  GIG  for  connectivity . 

Combat  system  automation 
capitalizes  on  the  rich  GIG 
information  and  knowledge 
base. 

Each  combat  system 
contributes  its  sensed 
information  to  the  GIG . 
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On  the  combat  system  side  is  a  continuing  quest  for  more  and  higher  quality  information  that  can  be  used 
to  guide  battle  decisions.  Our  combat  system  concepts  strive  toward  adaptable,  automated  capability, 
optimized  across  a  collection  of  cooperating  joint  assets  can  only  realize  their  potential  with  the  type  of 
adaptable,  ubiquitous,  and  improved  communications  infrastructure  promised  by  the  GIG.  Expanded 
automation  capability  appears  to  takes  a  central  position  in  connecting  these  two  objectives,  as  suggested 
above. 

The  richness  of  information  made  available  (and  readily  usable )  by  the  GIG  will  elevate  the  capabilities 
possible  in  a  generalized  automation  scheme.  No  longer  constrained  to  base  decisions  on  own  unit 
information,  generalized  decision  automation  will  be  free  to  identify  and  apply  best  sources,  to  incorporate 
information  on  other  units’  status  and  intent,  and  to  utilize  the  real-time  evolving  experience  base  of  its 
peers.  But  to  achieve  this  integrated  vision,  the  combat  systems  must  be  able  to  acquire  information  with 
the  accuracy,  reliability,  and  timeliness  required  for  making  real-time  warfighting  decisions.  Bridging 
these  two  simultaneously  evolving  communities  will  take  concerted  effort,  but  with  potentially  high 
payoff. 

The  development  of  our  NGCS  key  concepts  may  also  contribute  to  the  GIG’s  NCES.  The  development  of 
components  that  support  aggregation,  automation,  and  adaptation  might  well  be  candidates  for  an 
extension  of  the  current  nine  core  enterprise  services  of  the  NCES[1].  In  particular,  the  implementation  of 
aggregation  must  employ  a  standard  form  across  the  participating  systems  to  reap  the  desired  advantages. 

[1]  “Global  Information  Grid  (GIG)  Enterprise  Services  (GIG  ES)  Capability  Development  Document,”  in 
Defense  Acquisition  Guidebook ,  VI. 0  Section  7. 2.4.7,  17  October  2004. 
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Summary 

•  Drivers  for  change: 

-  Threat,  new  missions,  new  environment,  cost,  complexity 

•  Concepts  for  the  future: 

-  Aggregation,  automation,  adaptation 

-  Operational  control,  enhanced  awareness/understanding, 
communications  infrastructure 

•  Relationship  to  the  GIG 

-  GIG  provides: 

•  More  information  for  better  decisions 

•  Infrastructure  for  force  level  collaboration 

-  NGCS  provides: 

•  An  “actor”  environment  for  the  GIG  information/knowledge 

•  Aggregation  concepts  and  service  definitions 

0500493_UK.ppt-1 6 


Summary 

The  authors  have  assembled  a  summary  of  factors  driving  Navy  combat  systems  to  change.  A  synthesis  of 
drivers  and  operational  vision  led  to  a  collection  of  primary  and  secondary  concepts  that  establish  general 
system  capabilities  that  can  be  applied  across  a  wide  variety  of  military  systems  and  missions.  The  primary 
concepts  of  aggregation,  automation,  and  adaptation  are  discussed  in  moderate  detail  and  applied  in 
“thought  exercise”  form  to  the  BMD  mission.  While  these  concepts  have  emerged  from  a  Navy  context, 
they  should  be  equally  appropriate  to  the  combat  systems  of  the  other  services.  Indeed,  without  adoption  of 
compatible  (if  not  identical)  concepts  by  the  developers  of  all  joint  future  combat  systems,  the  complex 
collaborative  system  behavior  envisioned  will  not  occur.  Because  a  broad  set  of  systems  would  be 
integrated  and  controlled  through  these  combat  systems,  it  seems  highly  beneficial  to  address  these 
concepts  with  general  implementations  that  can  be  universally  applied. 

The  development  of  the  GIG  is  a  highly  relevant  effort  to  the  evolution  of  combat  systems  and  specifically, 
to  the  described  concepts.  While  it  will  require  time  to  align  objectives,  requirements,  and  communities,  it 
would  seem  inevitable  that  the  significant  capabilities  present  in  current  combat  systems  must  eventually 
benefit  from  the  rich  information  environment  that  will  be  assembled  in  the  GIG 
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6-DoF 

AAW 

BMC3 

BMCS 

BMD 

C2 

C2BM 

CS 

GBI 

GIG 

GWOT 

HLS 

IR 

NCES 

NGCS 


Acronyms 


Six-Degree  of  Freedom 
Anti- Air  Warfare 

Battle  Management  Command,  Control  and  Communications 

Battle  Management  Control  System 

Ballistic  Missile  Defense 

Command  and  Control 

Command  and  Control/Battle  Management 

Combat  System 

Ground-Based  Interceptor 

Global  Information  Grid 

Global  War  on  Terrorism 

Homeland  Security 

Infrared 

Net-Centric  Enterprise  Services 
Next  Generation  Combat  System 
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Next  Generation  Manufacturing 
Technology  Initiative  and  the  Model- 

Based  Enterprise 

NDIA  Systems  Engineering  Conference 
San  Diego,  California 
October  26, 2005 


Richard  Neal  -  IMTI 
Gerry  Graves  -  ATI 
Leo  Reddy  -  NACFAM 


NACFAM [ _ 
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Analyst  for  Science 
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Discovery 
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!n start!  Access  to 
Exactly  What  YOU 
Seed  to  Know 


A  Rich  Set  of 
Tech  n  0109  v 
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Comprehensive 
Manufacture  9 
Technology 
Management  Plans 


Dramatic 
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ROl  From  YOUR 
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The  NGMTI  Team 
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Three  non-profit  organizations  with  strong  expertise 
and  experience  in  facilitating  collaborations. 

*  IMTI:  a  technology/research  management 
organization  with  a  mission  to  support  the 
nation’s  manufacturing  infrastructure 

*  NACFAM:  a  long-term  builder  of  leadership-level, 
nationwide  manufacturing  technology  public- 
private  partnerships 

*  ATI:  a  deeply  experienced  manager  of  advanced 
manufacturing  technology  research 
collaborations. 
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“NGMTI  is  dedicated  to  transforming  the 
U.S.  manufacturing  base  through 
technology  driven  innovation ” 


Importance  of  Manufacturing 
to  Innovation 
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❖  Drives  innovation:  Manufacturers  invest  $135  billion  annually 
in  R&D,  which  is  70%  of  industry  R&D  investment  and  more 
than  all  federal  R&D 

❖  Innovative  mfg  process  technologies  are  the  most  effective 
means  to  reduce  China’s  low-wage  advantage 

❖  Yet  industry  gives  low  priority  to  process  technologies  and  is 
moving  R&D  offshore 

❖  Only  2%  of  federal  $132  billion  R&D  budget  spent  on  basic  and 
applied  manufacturing  tech 

❖  Manufacturing  R&D  has  never  been  a  White  House  “Grand 
Challenge” 
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The  NGMTI  Solution 


Cnext 

GENERATION 

MANUFACTURING  TECHNOLOGY  INITIATIVE 


*  Provides  a  mechanism  for  building  and  executing 
an  innovative  manufacturing  R&D  strategy  for  both 
economic  growth  and  national  security  goals 

*  Represents  a  sustainable  organization  meeting 
critical  success  factors:  strategic  planning, 
industry-government  collaboration,  national  tools 

*  Coordinates  research  and  development  projects 
focused  by  strategic  investment  plans 

*  Leverages  university,  federal,  industrial  labs,  and 
research  consortia  nation-wide 


NGMTI  Provides  for  Future 
Common  Needs 


♦> 


Provide  breakthroughs 
that  produce 
transformational 
technologies 


Common 
Solutions  to 
Common 
Needs 


Provide 
technologies 
that  improve 
affordability 
and  sustainability 


❖  Create  innovative  opportunities  for  fast 
response  manufacturing  of  new  products 


|k|  BYT  GENERATION 

•##  I1EAI  MANUFACTURING  TECHNOLOGY  INITIATIVE® 


Government 

Funding 


Research  | 
Partners 


Industry- 

Government  Forum 


Breakthroughs 


industry 


|  STRATEGIC 
INVESTMENT 

PLAN 

|*  Thrust  Areas 
Project  Plans 

1  | 

Funding 


Communities  of  Practice 


TestNet 
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■  MANUFACTURING  TECHNOLOGY  INITIATIVE™ 


GENERATION 

MANUFACTURING  TECHNOLOGY  INITIATIVE 


Implementation/Transition  Plan 


-  Current  State  Assessment 

-  Future  State  Vision 
•  Goals  8i 

Requirements 

-  Notional  Timeline 


*  Value  to  Industry,  *  Single  or  Multiple 

DoD,  Nation  Related  Goals  from 

-  Value-Based  Prioritization  Roadmap 

■  Risk,  Readiness,  Return  *  Suitable  for  Single 

Project 


Generate 
White  Papers 

'  Challenge  &  Approach 


Draft  SOW 

Benefits  &  Business  Case 


Project  Plan 

Risk/Readiness 

Assessment 


*  Validate  Challenge  & 
Approach 

-  Refine  SOW  Sc  Plan 

*  Begin  "Engagement" 
Process 


■  Form  Teams 
-  Finalize  SOW  &  Plan 
- Secure  Funding  (Federal 
Agency  &  Industry 
Investment) 


-  industry  Collaborative 

•  Focused  Corporate  R&D 

•  Govt-Funded  (e  g,,  SB1R) 

•  Leverage  NGMT1  TestNet 


*  NGMT1  Experiments 
&  Pilots 

*  Industry  Commercialization 

*  Technology  Deployment  to 
Defense  Industrial  Base 
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NGMTI  Thrust  Areas 

+  Emerging  Process  Technologies 

+  Model-Based  Enterprise 

+  Safe,  Secure,  &  Reliable  Manufacturing 
Operations 

+  Enterprise  Integration 
+  Intelligent  Systems 
*  Knowledge  Management 
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Model-Based  Enterprise  Prioritization 


Roadmap  contains  KT  compilation  of 

80  Goals  w  300  Analysis  (  T~r  on /^i  Jmportant  Themesr  15  White 

Requirements  *  Papers 


Web-Based 

Search 


Cross  Cutting 


Topic  Specific 

•  Product  Realization 

•  Resource  Management 

•  Strategic  Management 


Literature 


J 


Project  Roadmaps 
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Model-Based  Enterprise 
White  Papers 
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❖  Flexible  Representation  of  Complex  Models 

❖  Shared  Model  Libraries 

System-of-Systems  Modeling  for  the  Model-Based  Enterprise 
Enterprise-Wide  Cost  Modeling 
Intelligent  Models 

Configuration  Management  for  the  Model-Based  Enterprise 
Product-Driven  Product  &  Process  Design 
Model-Based  Product  Life-Cycle  Management 
Model-Based,  Real-Time  Factory  Operations 
Model-Based  Distribution 
Multi-Enterprise  Collaboration 

❖  Model-Based  Resource  Management 

❖  Information  Delivery  to  Point  of  Use 
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Emerging  Process  Technologies 


600  +  Technologies 

Expert  Screen  _ 

_ - _ *120  Significant 

Technologies 


r\ 

Previous 

Workshops 

V 

Interviews 

Cross  Cutting 

> 

Web-Based 

Topic  Specific 

Search 

Literature 

J 


1-20  White 
Paper  Topics 


► 


Project  Roadmaps 
and  Investment 
Plans 
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EPT  White  Papers 
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❖  Low-Cost  Titanium  Powder  Production 

❖  High-Frequency  Laser  Machining 
Friction  Stir  Joining  Technologies 

Improved  Thin-Film  Processes  for  Semiconductor  Fabrication 
Microreactors  &  Processing  Methods 
Digital  Direct  Manufacturing 

Affordable,  Lightweight  Large  Structural  Composites  Manufacturing 
Nanomaterials  for  Glass  Coatings 
Smart,  Reconfigurable  Multifunction  Machine  Tools 
Thin-Film  Coatings  for  Paint  Elimination 
Manufacturing  Applications  for  Carbon  Nanotubes 
Advanced  Aerospace  Casting  Processes 
Precision  Optical  Finishing 

❖  Hybrid  Bearing  Manufacture 

❖  Military  Fuel  Cell  Technology 
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NGMTI  Current  Status 


*  28  project  plans  developed  for  MBE 
and  EPT,  with  “High-interest”  from 
both  defense  and  commercial  firms 

*  Project  teams  now  being  formed  for  13 
of  the  White  Paper  topics 

*  MBE  Forum  being  planned  for  the  fall 
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The  NGMTI  Thrust  Areas 
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❖  Model-Based  Enterprise 

❖  Emerging  Process  Technologies 

❖  Safe,  Secure,  Reliable,  and  Sustainable 
Manufacturing  Operations 

Enterprise  Integration 

Intelligent  Systems 

❖  Knowledge  Applications 
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Model-Based  Enterprise: 

A  Single  Objective 

❖  MBE  -  an  integrated  digital  environment  for 
addressing  all  aspects  of  the  enterprise 

❖  Requires  total  sharing  of  information  between  all 
elements  of  the  enterprise. 

New  approaches  and  toolsets  are  required 


Prioritization  to  Establish 
What  to  Do,  When 
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Model-Based  Enterprise:  - 

The  Views 


Business 

Management 


Product  Realization 


Enterprise 

Integration 


Enterprise 

Management 
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Such  an  Enterprise  Will  Be. . . 


Thanks  to  the  NNSA  for  sharing  jointly  developed  visuals  and  concepts! 
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Totally  Connected 
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An  Integrated  Seamless  Flow  of  Information  and  Knowledge 
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Knowledge  Rich 
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Product  Design  &  Engineering 


Manufacturing  Operations 


Shared 

Knowledge 


Process  Design  &  Engineering 


Production 

Planning 


Inspection,  Acceptance 
&  Certification 


Product  Assurance 


cross  the  life  cycle 
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Simulation  Based 
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Requirements 

Analysis 


Performance 

Assessment 


Instantly  Responsive  with  . . . 


mM 


Producibility 


Material  Behavior 


One-click  access  to  all  needed  analysis  capabilities 
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Capable  of  Supporting  Closed-Loop  Operation 
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Product  &  process  definition 
Specs  &  control  parameters 

Resource  &  schedule 
requirements 


As-Built 


As-Designed 


•  As-built  configuration  &  properties 

•  Process  performance  &  material  behaviors 


Digital  feedback  deepens  the  knowledge  base  for  future  products 
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Bottom  Line  . . . 
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In  a  totally  managed  enterprise  „  ,  ,  „  , 

Validated  Products 
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MBE  Roadmap  Process 
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❖  Define  the  current  state  of  MBE  capabilities 

❖  Develop  MBE  vision 

❖  Express  vision,  goals  &  requirements  in  strategic 
investment  roadmap  document 

❖  Establish  priorities 

♦  “Readiness,  risk  &  return” 

•  “Scope,  magnitude,  vital  to  US  competitiveness” 
Prioritize  with  Kepner-Tregoe  decision-making  tool 
Write  white  papers  on  critical  topics 

❖  Review  and  validation  by  TAP 

❖  Refine  white  papers 


27 


Narrowing  MBE  Focus 


Roadmap  contains  KT  compilation  of 

50  Goals  W /  247  Analysis  Top  20  Goals lmportant  Themesr  15  White  Paper 

Requirements  Topics 


Previous 

Workshops 


Interviews 


Web-Based 

Search 


Literature 

- - -  J 


Cross  Cutting 


Topic  Specific 


•  Product  Realization 

•  Resource  Management 

•  Strategic  Management 


Project  Roadmaps 


TAP 

Review 


12  Updated  Topics 
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Configuration  Management 
for  the  Model-Based  Enterprise 
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Objective:  Develop  an  integrated  system  that  assures  association 
of  the  right  information  with  any  product  or  process 
throughout  its  life  cycle. 

Benefits: 


Integrated 
Product  Model 


As-Designed  -- 

■  Product  &  process  definition 
*  Specs  &  control  parameters 

■  Resource  &  schedule 
requirements 


As-Buitt 

*  Configuration  A 
properties 

*  Process  performance 
<3  material  behaviors 


Association  of  correct  info 
with  each  version  of  each 
product  or  process  in  the 
enterprise 

Feedback  loop,  which 
enables  continuous 
product  improvement. 

Assured  ability  to 
reproduce 
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Flexible  Representation 
of  Complex  Models 


Cnext 

GENERATION 

MANUFACTURING  TECHNOLOGY  INITIATIVE 


Objective:  Develop  capability  to  create  collaborative  models  rich 
enough  to  support  all  MBE  functions. 


Benefits: 

•  Enables  full  evaluation 
of  any  decision 

•  Procurement  cost 
savings  in  the  billions  of 
dollars 

•  Reduced  time  to  market 

•  Reduced  costs 

•  Better  quality  products 
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System-of-Systems 

Modeling  for  Model-Based  Enterprises 


Objective:  To  develop  capabilities,  approaches,  and  tools  for 
integrated  multi-level,  multi-system  modeling  of 
products,  processes,  and  life-cycle  functions. 


Benefits: 

•  Composable  and  decomposable  models  enable  evaluation  of  total  system 
performance  within  its  operational  context 

•  Extends  SoS  philosophy  to  manufacturing  enterprise 

•  Enhanced  ability  to  simulate,  with  high  fidelity,  the  effects  of  wear  and  tear 
on  complex  systems  in  combat  and  training 
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Intelligent  Models 
for  Manufacturing 
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Objective:  Develop  intelligent  models  that  understand,  seek  out, 
acquire  knowledge  needed  to  execute  their  functions. 


Process 

Engineering 


Benefits: 

•  Dramatic  cost  savings 
through  elimination  of 
design  iterations 

Improved  logistics 
support  for  weapons 
systems 

Significant  reduction  of 
design  cycle  times 


Manufacturing 

Execution 


Mechanical 'Electrical/ 
Systems  Engineering 


Acceptance  & 
Certification 


Intelligent 

Product 

Model 


Requirements 

Analysis 


Life-Cycle 
Service 
&  Support 


Business 

Decision 

Support 
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Model-Driven  Product 
and  Process  Development 
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Objective:  Develop  simulation  capabilities  enabling  the  product 
model  to  fully  support  down  stream  operations. 


Benefits: 

-  Saves  money  and  assures 
product  quality 

-  Optimizes  use  of  product  and 
process  capabilities 

-  Reduces  the  extent  and  level 
of  design  changes 

-  Enhances  risk  analysis  and 
mitigation 
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Model-Based  Product 
Life-Cycle  Management 
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Objective:  Provide  the  capability  to  create  and  apply  hi-fidelity, 
scaleable  product  life-cycle  models. 


Benefits: 

•  Provides  a  toolset  for 
modeling  and  understanding 
life-cycle  cost  and 
supportability  impacts  . 

•  Enables  feed  back  from 
down-stream  experience  to 
improve  up-stream  functions. 

•  Improved  speed  and 
accuracy  of  technical  and 
business  decisions  over  the 
life  cycle, 

•  Ability  to  analyze  and 
reverse-engineer  “as-worn” 
parts  to  predict  failure 


Suppliers 
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Information  Delivery  to 
Point  of  Use 
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Objective:  Deliver  information  to  any  location  in  support  of  any 
enterprise  function 


Benefits: 

•  Largely  graphical 
information  delivery 

•  Job  compatible  delivery 

•  Graphical  format  saves 
money  in  multi-lingual 
support 

•  Reduced  warrantee  cost  for 
returns  due  to  fewer 
mistakes 
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MBE  Enablers  for  the 
Electro-Mechanical  Industry 
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Objective:  To  apply  product  and  process  models  to  define  and 
manage  all  enterprise  processes,  and  by  applying  science-based 
analytical  tools  to  make  optimal  decisions  at  every  step  of  the 
product  life-cycle. 


Modules*: 

Machine  Program 
Assy  and  Test 
Global  Manufacturing 
Library 
ERP/MES 

Verification  Execution 
LRI P/Full  Production 


Enterprise  Model  is  conWjsed  of  various  Modjrfes  and  Interfaces, 
residing  in  a  process  man^qement  architect! fire  based  on  business  needs. 

izatiprfcapabillity  increases 
erprise. 


Benefits: 

•  Model-Based  testing 
offers  development  time 
savings  of  50% 

•  Elimination  of  the 
“disconnect”  between 
development  and 
production 

•  Rapid  response  to 
customer  demands 
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Shared  Model  Libraries 
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Objective:  Enable  centralized  access  to  modular  components  to 
support  all  MBE  functions  and  optimize  enterprise 
decisions 


Benefits: 


Kequirements 

Analysis 


Event  Physics 


Material  Behavior 


Producibllity 


Provides  a  core  set  of 
models  affordable  and 
available. 

Reduction  in  cycle  time 
and  cost  by  up  to  40% 

Rapid  integration  and 
virtual  testing  of  complex 
weapon  systems 

Elimination/Reduction  of 
redesign/rework  costs  and 
time 
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Enterprise-Wide  Cost 
Modeling 
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Objective:  Provide  the  ability  to  model  and  predict  cost  for  every 
element  and  from  every  source  in  the  enterprise, 
including  uncertainty  and  risk. 


Vision  | 


New 

Strategic  Capital  PV  of 
Plan  cost  Dsn  Costs 


Cost  of 
Overhead 
•••  Components 


Request 


Materials  Effort 

Traditional 


Overhead 

Rate 


Benefits: 

•  Visibility  of  the  cost 
impacts  of  design 
changes 

•  Eliminating  low-ball 
estimates  with  directly 
traceable  sources 

•  Significant  areas  of  cost 
and  expense  can  be 
easily  identified 

•  Enables  evaluation  of 
Strategic  options 
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Model-Based  Real-Time 
Factory  Operations 
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Objective:  To  develop  enabling  technologies  for  real  time, 
model-based  control  of  factory  operations. 


Benefits: 

•  First  and  every  product 
correct  due  to  process 
control. 

•  Maximum  use  of 
production  capability 

•  More  efficient,  responsive, 
flexible,  and  capable 
manufacturing  base 

•  Shortened  timelines  to 
ramp  up  production 
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Model-Based 

Distribution 
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Objective:  Provides  a  framework  for  supporting  design  for  distribution 
planning,  execution,  and  re-planning. 


Benefits: 

•  “Engineer  out”  problems  in 
new  products  rollout 

•  Accommodates  far  more 
variables  in  distribution 
planning 

•  Improved  downstream  life- 
cycle  management 

•  Enables  definitive  information 
about  where  it  should  be 

•  Focuses  for  closing  the  loop 
on  where  it  is 


40 


Model-Based  Resource 
Management 


Cnext 

GENERATION 

MANUFACTURING  TECHNOLOGY  INITIATIVE 


Objective:  Create  a  cost  effective,  integrated  capability  for  evaluating 
options  and  directing  control  over  all  manufacturing 
resources.  Modular  and  easily  integrated  are  key  attributes. 


Benefits: 


Prccunt 

Models 


Manpowei 
Shills  „ 


Mater:  a  I 


Dparatj’dins 

Models 


Process 

Models 


-ct  &  Process 
Design 


inve 


Sales  £ 
Distribution 


Service  & 
Supoort 


Prod  uct/P  recess  Manufacturing 

Development  O  perations 


Provision  of  model-based  resource 
management  capabilities  that: 

•  Greatly  reduce  the  cost  of 
acquiring,  deploying  and 
maintaining  a  resource 
management  system 

•  Enable  far  greater  accuracy 
and  efficiency  in  managing 
resources 

Enhanced  ability  of  smaller 
suppliers  to  choose  resource 
management  tools,  and  to 
interface  with  prime  manufacturers 


Multi-Enterprise  Collaboration 


Cnext 

GENERATION 

MANUFACTURING  TECHNOLOGY  INITIATIVE 


Objective:  Provide  a  tool  set  to  support  multi-enterprise  collaboration 


Benefits: 


Company  A 


Shared 

Information 


Shared 

Information 


Company  D 


Manufacturing 

Knowledge 

Repository 


Shared 

Information 


Shared 

Information 


Company  C 


Mitigates  the  cost  of 
transferring  or  recreating 
design  definitions  shared 
among  different  members 
of  the  supply  chain. 

Enables  ability  to 
objectively  evaluate 
potential  suppliers 

Reduces  contract 
administration  costs  by 
50%  through  integrated 
reporting  and  management 
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Summary 


Cnext 

GENERATION 

MANUFACTURING  TECHNOLOGY  INITIATIVE 


❖  NGMTI  is  an  important  program  to  the 
nation 

❖  We  are  off  to  a  fast  start  and  making  great 
progress 

❖  Project  formation  is  in  full  swing  - 
opportunity  knocks 


Ngmti.org 
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C-17:  A  High  Performance  Program 


MEETING  OUR  COMMITMENTS 


J  Excellent  Quality 
J  Ah ea d  of  Sch edul  a 
J  On  Price 

J  180  Aircraft  Program 


MEETING  OUR  COMMITMENTS 


J  138  US  A?  Aircraft  -  6  Bases 
J  Worldwide  Operations 
J  Best  Fleet  Reliability 
J  4  UK  C-1 7s  Delivered 


Over  898, 750  Flight  Hours! 

USAF  Fleet  -  872,885  UK  Fleet  -  23,085 


C-17  Awards 


1994 

Collier  Award 


1996  California 
Quality  Award 


1998  Malcolm  2001  UK  MOD  Smart 
Baldrige  National  Acquisition  Award 
Quality  Award 


2003  Missouri  State 
Quality  Award 


2001  ISO900 1-2000  / 
AS9100A  Certification 


CMMI 


SEI 
Standard 
Level  5 


2003  Georgia 
Oglethorpe 
Award 


2002  iW  Winner:  2002  California  Awards 

Top  10  Best  Plants  for  Performance  Excellence 

(Gold  &  Silver) 


2003  Governor’s  Award  For 
Performance  Excellence 


Quality  Journey 


/ 


Malcolm 

Baldrige 

Applicant 


CCE 
Team 
Excellence 
Applicant 


Best 

Practice 

Exchange 


Quest  for 
Excellence 
and 

speaking 

engagements 


CAPE* 

Applicant 


Internal 

Assessment 


Internal 

Assessment 


Industry 
Week 
Top  25 
Finalist 


CCE 

Team 

Excellence 

Applicant 


ter/ jn*r*a£  .  ’  - 


Internal 

Assessment 


Industry 
Week 
Top  10 
Finalist 


Trained 


Use  CAPE 
feedback  to 
improve  using 
internal 


Deployment 

of  best  Incorporation  developed 
practices  of  high-leverage  in-house 


Examiners  Examiners 


to  new 


tenants 


actions  into 
strategic 
planning 


Uti!  J  I  (•Till  filiffi 


(tt 


AFRL  Systems  Engineering  Initiative 

Risk  Management  for  Science  and  Technology 


October  24  -  27,  2005 


Bill  Nolte 


Electronics  Engineer 

Col  Norman  Anderson 
Chief  Engineer,  Space  Vehicles 

Bob  McCarty 

Systems  Engineering  Lead 
Air  Force  Research  Laboratory 


Technology 
Life  Cycle 

AFRL  SE 
Initiative 

Risk 

Management 

Interim 

Conclusion 

Tools 

Conclusion 


Technology  Life  Cycle 
The  Whale  Chart 


Concept  System  Development  Operations  S 

Relinement  &  Demonstration  Support 

Technology  Productions 

Development  Deployment 

TRL 12345  67  8  9 


Basic  Research 

Applied  Research 

Advanced  Research 
ATD 


•The  Whale  Chart  maps  the  Life  Cycle  to  the  Readiness  Levels  and 
R&D  Stages 

•A  technology’s  usefulness  changes  over  time 

Utility  increases  as  a  technology  matures 

Utility  decreases  as  a  technology  becomes  obsolete 


Technology 
Life  Cycle 

AFRL  SE 
Initiative 

Risk 

Management 

Interim 

Conclusion 

Tools 

Conclusion 


Knowledge  Growth 


Key  Question 

Basic 

Research 

Applied 

Research 

Advanced 

Research 

ATD 

Man 

Tech 

1.  Who  is  your  customer? 

Partial 

Nearly 

Complete 

Complete 

Complete 

Complete 

2.  What  are  customer's  requirements? 

Partial 

Partial 

Nearly 

Complete 

Complete 

Complete 

3.  How  will  you  demonstrate  you  have 
met  the  requirements? 

Partial 

Partial 

Nearly 

Complete 

Complete 

Complete 

4.  What  are  the  technology  options? 

Extremely 

Limited 

Nearly 

Complete 

Complete 

Complete 

Complete 

5.  Which  is  the  best  approach? 

Extremely 

Limited 

Nearly 

Complete 

Complete 

Complete 

Complete 

6.  What  are  the  risks  to  developing  the 
selected  technology? 

Partial 

Partial 

Nearly 

Complete 

Complete 

Complete 

7.  How  will  you  structure  your  program 
to  meet  requirements  and  manage  risk? 

Partial 

Nearly 

Complete 

Complete 

Complete 

Complete 

8.  What  is  your  business -based  transition 
plan  that  meets  customer  approval? 

Extremely 

Limited 

Partial 

Nearly 

Complete 

Complete 

Complete 

3 

Key  Questions  and 
Systems  Engineering 


Technology 
Life  Cycle 

AFRL  SE 
Initiative 

Risk 

Management 

Interim 

Conclusion 

Tools 

Conclusion 


Gov’t 

Lead 


Industry 

Lead 


Requirements 

Analysis 


Who  is  your  customer? 
What  are  customer’s 
requirements? 


Balance  & 
Control 


What  are  the 
technology  options? 


Functional  Analysis  / 
Allocation 

I 


How  will  you 
structure  your 
program  to  meet 
requirements 
and  manage 
risk? 


How  will  you  demonstrate 
you  have  met  the 
requirements? 

T 


Which  is  the 
best  approach? 


What  are  the  risks  to 
developing  the 
selected  technology? 


Key  questions  provide  the  loops  & 
Balance  &  Control  mechanisms  of 
the  Systems  Engineering  process! 


Solution 
Synthesis 

What  is  your  business-based 
transition  plan  that  meets 
customer  approval? 


Technology 
Life  Cycle 

AFRL  SE 
Initiative 

Risk 

Management 

Interim 

Conclusion 

Tools 

Conclusion 


R&D  Focus  on  Risk 


Two  of  the  Key  Questions  Focus  on  Risk  in  R&D 


What  are  the  risks  to  developing 
the  selected  technology? 

How  will  you  structure  your  program 
to  meet  requirements  and  manage  risk? 
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Technology 
Life  Cycle 

AFRL  SE 
Initiative 

Risk 

Management 

Interim 

Conclusion 

Tools 

Conclusion 


RM  Tailored  to  R&D  Goals 


*  Three  Distinct  Levels  of  Research  and 

Development 

—  Basic  Research  -  develop  a  fundamental 
understanding  of  selected  physical 
properties 

—  Applied  Research  -  investigate  application 
of  physical  properties  to  selected  technical 
needs 

—  Advanced  Technology  Development  - 
explore  application  of  technology  to  assess 
military  relevance 
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Technology 
Life  Cycle 

AFRL  SE 
Initiative 

Risk 

Management 

Interim 

Conclusion 

Tools 

Conclusion 


Philosophy  of  RM  in 
Basic  Research 

What 

*  Develop  cost  estimates  for  advancement  of  technology 
to  useful  level 

*  Identify  development  options  and  relative  difficulty  of 
options 

*  Maintain  budget  within  pre-defined  boundaries 

How 

*  Establish  knowledge  incremental  goals 

*  Estimate  cost/time  needed  to  achieve 

*  Determine  risks  associated  with  maintaining 
cost/schedule 

*  Track  variances  for  periodic  cost/schedule  replan 

Primary  purpose  of  RM  in  Basic  Research  is  to  refine  development  roadmap 


Philosophy  of  RM  in 
Applied  Research 


What 


Technology 
Life  Cycle 

AFRL  SE 
Initiative 

Risk 

Management 


Develop  technology  into  a  repeatable  engineering 
capability 

Identify  extent  of  applicability  of  technology  to  military 
needs 


Interim 

Conclusion 

Tools 


Determine  the  cost/benefit  parameters  of  this  new 
caapability 

How 


*  Explore  range  of  application  of  technology 

*  Refine  development  roadmap  for  specific  applications 

*  Determine  risks  associated  with  achieving  required 
performance  at  known  cost/schedule 

*  Identify  issues  of  repeatability  and  define  mitigation 

approaches _ 

Primary  purpose  of  RM  in  Applied  Research  is  to  balance  cost  &  performance 


Philosophy  of  RM  in 
Advanced  Technology  Development 


What 


Technology 
Life  Cycle 

AFRL  SE 
Initiative 

Risk 

Management 


Apply  engineering  capability  to  specific  military  need 
Identify  issues  causing  uncertainty  in  application 
Refine  cost/performance  relationship. 


Interim 

Conclusion 

Tools 

Conclusion 


How 

Manage  to  cost/schedule 

Provide  mitigation  options  and  go/nogo  gates 

Determine  risks  early,  maintain  constant  awareness 

Identify  potential  of  cost/schedule  failure  early 
(precursors),  manage  proactively 


Primary  purpose  of  RM  in  ATD  is  to  balance  cost,  performance,  schedule 
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Technology 
Life  Cycle 

AFRL  SE 
Initiative 

Risk 

Management 

Interim 

Conclusion 

Tools 

Conclusion 


Risk  Management  Summary 


Key  Questions  6  and  7  provide  the  basis  of  the  AFRL  Risk 
Management  process 

Questions  apply  to  R&D  programs  at  all  stages  of  maturity 

Knowledge  available  to  the  program  manager  changes  with 
program  maturity 

Risk  Management  philosophy  changes  with  program  maturity 
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Technology 
Life  Cycle 

AFRL  SE 
Initiative 

Risk 

Management 

Interim 

Conclusion 

Tools 

Conclusion 


Risk  Management  Tools 


Disclaimer: 

This  is  a  partial  listing  of  risk  management  tools  that  have 
proved  to  be  useful  in  the  science  and  technology 
environment 

The  presence  of  a  tool’s  name  and  description  in  this 

presentation  does  not  constitute  an  endorsement  by  the 
US  Air  Force  or  any  of  its  officers  or  personnel 

The  absence  of  a  tool’s  name  and  description  from  this 

presentation  does  not  constitute  a  finding  of  unsuitability 
or  a  criticism  of  the  product  by  the  US  Air  Force  or  any  of 
its  officers  or  personnel 
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Technology 
Life  Cycle 

AFRL  SE 
Initiative 

Risk 

Management 

Interim 

Conclusion 

Tools 

Conclusion 


Risk  Management  Tools 


AFMC/TRIP  Risk  Mgmt 


Active  Risk  Manager  (ARM) 


IPPD  Control  Suite 


Probability  /Consequence  Screening  (P/CS) 


Risk  Matrix 
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RiskNav 


Risk  Management  Tools 


Technology 

Life  Cycle 

AFRL  SE 
Initiative 

Risk  Radar 

Risk  Radar  Enterprise 

Risk 

Management 

Interim 

Technical  Risk  Identification  &  Mitigation  System 
(TRIMS) 

conclusion 

Tools 

@Risk 

Conclusion 

Consolidated  Risk  Assessment  Methodology 
(CORAM) 

Risk  Matrix 
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Risk  Management  Tools 


Technology 
Life  Cycle 

AFRL  SE 
Initiative 

Risk 

Management 

Interim 

Conclusion 

Tools 

Conclusion 


Pertmaster 


Risk  + 


Crystall  Ball 


Dynamic  Insight 


Active  Risk  Manager 


Risk  Nav 


Microsoft  Excel  user  created  applications  can  also  be  useful 

RiskHammer 
TRL  Calculator 
FMEA 
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Technology 
Life  Cycle 

AFRL  SE 
Initiative 

Risk 

Management 

Interim 

Conclusion 

Tools 

Conclusion 


Summary 

The  AFRL  Systems  Engineering  Initiative  is  a 
method  of  managing  risk  in  Science  and 
Technology 

Applicable  early  in  the  technology  life  cycle 

Key  questions  test  risk  management  during 
program  reviews 

A  variety  of  risk  management  tools  exists 
COTS 

User  created  applications 
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Discussion  /  Questions 
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Definition 

•  Technical  performance  measures  (TPMs)  are  tools  that 
show  how  well  a  system  is  satisfying  its  requirements  or 
meeting  its  goals 

•  TPMs  provide  assessments  of  the  product  and  the  process 
through  design,  implementation  and  test 

•  TPMs  are  used  to: 

-  forecast  values  to  be  achieved  through  planned  technical 
effort 

-  Provide  visibility  of  actual  versus  planned  performance 

-  Provide  early  detection/prediction  of  problems  requiring 
management  attention 

-  Support  assessment  of  the  impact  of  proposed  changes 

•  determine  the  impact  of  these  differences 

•  trigger  optional  design  reviews 


BAE  SYSTEMS 


TPM  examples 

*  Reliability 

*  Power  required 

*  Weight 

*  Throughput 

*  Human  Factors 

*  Response  time 

*  Complexity 

*  Availability 

*  Accuracy 

*  Speed 
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BAE  SYSTEMS 


Requirements  Criteria  for  TPMs  creation 

•  High  priority  requirements  that  have  an  impact  on 

-  mission  accomplishment 

-  customer  satisfaction 

-  cost 

-  system  usefulness 

•  High  risk  requirements  or  those  where  the  desired 
performance  is  not  currently  being  met 

-  the  system  uses  new  technology 

-  new  constraints  have  been  added 

-  the  performance  goal  has  been  increased 

-  but  the  performance  is  expected  to  improve  with  time 

•  Requirements  where  performance  can  be  controlled 

•  Requirements  where  the  program  manager  is  able  to 
rebalance  cost,  schedule  and  performance 

•  TPMs  should  meet  all  of  these  characteristics 

•  Less  than  1%  of  requirements  should  have  TPMs 
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TPM  Characteristics 

*  Should  be  important  and  relevant 

*  Should  be  relatively  easy  to  measure 

*  Performance  should  be  expected  to  improve  with 
time 

*  If  the  measure  crosses  its  threshold,  corrective 
action  should  be  known 

*  The  measured  parameter  should  be  controllable 

*  Management  should  be  able  to  tradeoff  cost, 
schedule  and  performance 

*  Should  be  documented 

*  Should  be  tailored  for  the  project 


BAE  SYSTEMS 


Collecting,  Reporting  and  Displaying  TPM  data 

•Systems  Engineering  Manager  is  responsible  for 
collecting,  analyzing,  reporting  and  responding  to  TPM  data 

•TPMs  should  be  presented  to  the  person  who  can  do 
something  about  it.  Often  this  is  the  Chief  Engineer 

•Program  Manager  has  oversight 

•Measures  Analysis  Group  might  use  them  for  process 
improvement  suggestions 

•TPM  measures  can  be  displayed  with  graphs,  charts, 

diagrams,  figures  or  frames 

-  e.  g.  Statistical  Process  Control  Charts,  Run  Charts,  Flow 
Charts,  Histograms,  Pareto  Diagrams,  Scatter  Diagrams, 
Check  Sheets,  PERT  Charts,  Gantt  Charts,  Line  Graphs, 
Process  Capability  Charts  and  Pie  Charts 
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TPM  Measurement 

*  The  measuring  method  will  vary  with 
life-cycle  phase 

*  Start  with  legacy  systems,  blue  sky 
guesses  and  approximations 

*  Derive  data  from  models  and 
simulations 

*  Collect  data  from  prototypes 

*  Measure  data  on  rudiments  of  the  real 
system 


Typical  TPM  tracking  chart 
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BAE  NSS  TPM  Proceedure 

•  TPMs  are  established  during  the  proposal  process  by  the 
Chief  Engineer  for: 

-  Key  Requirements 

•  Customer’s  main  requirement  drivers 

•  System  requirements  important  to  the  Business 

-  Key  Functions 

•  System  level  functions  essential  to  the  performance  of  the  system 

-  Critical  Design  Features 

•  Represent  uncertainty  with  respect  to  confidence  in  the  design 
approach 

•  Represent  technical  risk  that  is  manifest  as  borderline  performance 

•  TPMs  are  approved  as  a  part  of  the  Engineering  Estimate 
Approval  process 

•  TPMs  are  maintained  in  the  program’s  risk  register 

•  The  TPM  Proceedure  is  PD0644 


The  BAE  TPM  Proceedure 


BAE  SYSTEMS 


•  CMMI  Requirements  Development  (RD)  process  area  (SP  3.3-1) 
covers  TPMs 

•  Related  CMMI  areas  include  Project  Monitoring  and  Control 
(PMC)  and  Risk  Management  (RSKM) 
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ID 

number 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 


Process  Step 

Identify  key  performance  requirements  (refer  to  the  Requirements  Analysis 
process  output).  These  are  candidate  TPMs. 

Categorize  key  requirements  within  the  requirements  management  tool  (e.g., 
DOORS).  Add  TPM  attributes  as  needed. 

Identify  critical  technical  parameters. 

Perform  risk  analysis. 

Select  TPMs  to  be  tracked  throughout  applicable  program  phase(s).  Determine 
frequency  of  reporting. 

Conduct  expert  review  of  selected  TPMs.  Feedback  results  and  update. 

For  each  TPM,  establish  upper  and  lower  limits  and  performance  growth 
values  for  discrete  reporting  points. 

Assign  responsibility  for  meeting  TPMs. 

Incorporate  TPMs  into  appropriate  program  documents  (e.g.,  PPofE,  SEMP, 
SDP,  DRB,  etc.). 

Use  the  project  risk  management  process  to  track  TPMs. 

Schedule,  collect  and  report  TPM  measurements. 

Perform  corrective  action  and  risk  mitigation  on  TPMs  that  do  not  meet 
performance  growth  values. 
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TPM  Collection 

*  TPMs  require  quantitative  data  to  evaluate  the 
likelihood  of  satisfying  the  system 
requirements 

*  Gathering  such  data  can  be  expensive 

*  Because  of  the  expense,  not  all  requirements 
have  TPMs,  just  the  high  priority  requirements. 
As  a  rule  of  thumb,  less  than  1%  of 
requirements  should  have  TPMs. 

*  A  TPM’s  values  change  with  time,  hopefully 
getting  closer  and  closer  to  the  goal 

*  TPMs  are  linked  to  a  requirement,  have 
quantitative  values  and  a  risk  level 
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Typical  TPM  ranking  table 


Technical 
Performance 
Measure  (TPM) 

Source 

Requirement 

Quantitative 

Performance 

Requirement 

Current  TPM 
Value 

Risk  of 
Not 

Meeting 

TPM* 

Image 

processing  time 
(minutes) 

ID  #123 

Less  than  5 
minutes  from 
time  of  request 

10  minutes 

1 

MTBF  of  system 

ID  #321 

Greater  than 

1000  hours 

750  hours 

3 

Availability 

(operational) 

ID  #  456 

98%  (minimum) 

95% 

2 

*1=  Very  High 
2=  High 
3=  Moderate 
4=  Low 
5=  None 
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Prioritization 

*  Requirements  are  prioritized 

*  In  addition,  TPMs  should  be  prioritized  with 
relative  importance  to  the  customer 

*  BAE  Systems  NSS  RF.PrioritizeRequirements 
documents  process  for  requirements 
prioritization  criteria  and  methods 
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TPM  prioritization 


TPM 

Planned  value 

Current  value 

Relative 

importance 

Image 
processing 
time  (sec.) 

30  sec.  max 

45  sec. 
for  an  SES 
simulation 

10 

Power 

required 

10  KV  max. 
UPS  2  hr. 
backup 

12  KV 

UPS  1.5  hr. 
Vendor  data 

8 

Weight 

600  lbs.  max 
man  portable 
modules 

625  lbs. 
six-modules 
CAD  mockup 

7 

1  =  least  important 
10  =  most  important 


TPMs  can  be  organized  hierarchically 

For  example 

*  System  lifetime 

-  mechanical  lifetime 

-  electrical  lifetime 

•  power  consumption 

•  battery  capacity 

*  The  lower  level  TPMs  (or  measures)  are 
used  to  derive  values  of  higher  level 
TPMs 

*  The  top-level  TPMs  may  be  reported  to 
Senior  Management 
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Tracking  TPMs 

*  The  DOORS  requirements  module  should  have  an 
attribute  named  TPM 

*  The  name  of  each  TPM  should  be  entered  in  the 
attribute  field  of  the  appropriate  DOORS 
requirement  and  this  should  be  linked  to  the  TPM 
module 

*  Each  TPM  should  also  be  referenced  in  the 
project’s  Risk  Register  and  be  evaluated  monthly 


DOORS 

Database 

- Link - 

TPM 

>  1 

Module 

- Link - 

Risk 

Register 
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Optional  Independent  Design  Reviews 

*  TPMs  can  also  be  used  to  trigger  optional 
independent  design  reviews 

*  Only  eight  design  reviews  are  mandated 

*  If  a  TPM  exceeds  its  thresholds,  then  an  optional 
independent  design  review  (IDR)  will  be  added  to 
the  Engineering  Plan 

*  PS0366  Plan  and  Conduct  Independent  Design 
Reviews 

*  PD0602  Plan  Independent  Design  Reviews 

*  PD0603  Conduct  Independent  Design  Reviews 
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The  big  picture 


•  Program  managers  tradeoff  cost,  schedule  and 
technical  performance  of  a  system 

•  Cost  and  schedule  are  often  tracked  with  an  earned 
value  system 


•  TPMs  give  managers  a  way 
performance 

•  Managers  can  adjust 
cost  and  schedule  per  TPM 
forecasts 


to  track  technical 


19 


BAE  SYSTEMS 


Solar  Oven  Case  Study 

*  As  an  example  of  using  a  TPM,  let  us  consider  the 
design  and  manufacture  of  solar  ovens 

•  In  many  societies  people  spend  as  much  as  50% 
of  their  time  acquiring  wood  for  their  cooking 
fires 

•  To  address  this,  people  have  been  designing  and 
building  solar  ovens 

*  Let  us  examine  the  solar  oven  design  and 
manufacturing  process  that  we  followed  in  a 
Freshman  Engineering  Design  class  (Engr-102)  at 
the  University  of  Arizona 
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Risk  analysis.) 

For  each  identified  risk,  students  recorded  the  Risk  Name, 
description,  impact,  probability,  type  and  risk  mitigation 
plan 

For  the  solar  oven  project  three  risks  were  identified 

Risk  One 
Name:  High  Cost 

Description:  Material  for  the  ovens  is  provided.  But  some 
students  paid  $100  for  special  materials  and  told  their 
parents  that  was  required 
Impact:  medium 
Probability:  low 
Type:  monitor 
Plan:  Compute  cost 
for  every  design 
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Risk  analysis  of  solar  oven2 

Risk  Two 

Name:  Failure  to  Have  Oven  Ready  for  Testing 

Description:  Everyone  must  test  at  the  same  time 
on  the  same  day.  If  a  team  is  not  ready,  they 
cannot  be  tested  fairly. 

Impact:  high 

Probability:  low 

Type:  manage 

Plan:  Require  final  design  7  days  before  scheduled 
test  date  and  require  preproduction  unit  3  days 
in  advance 
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Risk  analysis  of  solar  oven  3 

Risk  Three 

Name:  Insufficient  Internal  Oven  Temperature 
Description:  The  ovens  must  get  hot  enough  to 
bake  bread. 

Impact:  high 
Probability:  high 
Type:  resolve 
Plan:  Make  it  a  technical 
performance  measure 
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Design  the  TPM 

*  When  a  loaf  of  bread  is  finished  baking,  its 
internal  temperature  should  be  200°F 

*  To  reach  this  internal  temperature,  commercial 
bakeries  bake  the  loaf  at  445°F 

*  As  initial  values  for  our  oven  temperature  TPM, 
we  chose  a  lower  limit  of  212°F,  a  goal  of  445°F, 
and  an  upper  limit  of  520°F 

*  The  tolerance  band  shrinks  with  time  as  shown  in 
the  upcoming  figure 


25 


BAE  SYSTEMS 


TPM  template 

•  Name:  Internal  Oven  Temperature 

•  Purpose:  ensure  that  most  ovens  pass  the  scheduled  test 

•  Source  requirement:  assignment  for  Engr-102 

•  Risk  level:  resolve 

•  What  should  be  measured?  internal  oven  temperature  in 
degrees  Fahrenheit 

•  How  should  it  be  measured?  test 

•  How  often  should  it  be  measured?  daily 

•  During  which  project  phases  should  it  be  measured?  all 

•  How  should  it  be  displayed?  see  figure 

•  To  whom  should  it  be  presented?  Engr-102  instructor 

•  Threshold  above  or  below  which  action  is  necessary:  the 
lower  limit  shown  in  the  figure 

•  What  action  should  be  performed?  suggest  new  design  or 
negotiate  with  the  customer  to  relax  the  requirements 

•  Who  should  perform  this  action?  Engr-102  instructor 
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Improvement) 

*  In  the  beginning  our  day-by-day  measurement 
values  increased  because  of: 

-  finding  better  insulators, 

-  finding  better  glazing  materials  (e.g.,  glass  and  Mylar), 

-  sealing  the  cardboard  box  better, 

-  aiming  at  the  sun  better,  etc. 

*  At  the  time  labeled  “Design  Change-1,”  there  was 
a  jump  in  performance  caused  by  adding  a 
second  layer  of  glazing  to  the  window  in  the  top 
of  the  oven 

*  This  was  followed  by  another  period  of  gradual 
improvement  as  we  learned  to  stabilize  the  two 
pieces  of  glazing  material 
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lmprovement2 

*  At  the  time  labeled  “Design  Change-2,”  there  was 
another  jump  in  performance  caused 
incorporating  reflectors  to  reflect  sunlight  onto 
the  window  in  the  oven  top 

*  This  was  followed  by  another  period  of  gradual 
improvement  as  we  found  better  shapes  and 
positions  for  the  reflectors 
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Study  the  requirement 

*  We  might  not  attain  our  goal 

*  We  reevaluated  the  process  and  requirements 

*  “Consequences  of  insufficient  oven  temperature: 

-  Enzymes  are  not  deactivated  soon  enough,  and  excessive 
gas  expansion  causes  coarse  grain  and  harsh  texture 

-  The  crust  is  too  thick,  because  of  drying  caused  by  the 
longer  duration  of  baking 

-  The  bread  becomes  dry,  because  prolonged  baking 
causes  evaporation  of  moisture  and  volatile  substances 

-  Low  temperatures  cannot  produce  carmelization,  and 
crust  color  lacks  an  appealing  bloom 
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Alternatives 

*  If  the  dough  were  made  richer  by  adding  sugar, 
eggs,  butter  and  milk,  we  could  get  away  with 
temperatures  as  low  as  350°F 

*  But  we  decided  to  design  our  ovens  to  match  the 
needs  of  our  customers,  rather  than  try  to  change 
our  customers  to  match  our  ovens 
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Change  the  requirement 

*  After  consulting  some  bakers,  our  managers 
decided  that  375°F  would  be  sufficient  to  avoid 
the  above  problems 

*  Therefore,  the  requirements  were  changed  at  the 
indicated  spot  and  our  design  was  able  to  meet 
the  goal  of  the  TPM 

*  Of  course,  this  change  in  requirements  forced  a 
review  of  all  other  requirements  and  a  change  in 
many  other  facets  of  the  design 

*  For  example,  the  baking  duration  versus  weight 
tables  had  to  be  recomputed 
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Pilot  at  BAE  Systems 

*  In  2005,  a  mature  Archive  and  Dissemination  development 
program  piloted  our  TPM  process 

*  This  program  has  been  running  for  seven  years 

*  We  used  it  on  a  new  spiral  that  was  to  last  seven  months 
from  funding  to  delivery 

*  TPMs  were  selected  for  less  than  1%  of  the  program’s  7000+ 
system  requirements 

*  The  selected  TPMs  were  related  to  image  processing  and 
data  export  (dissemination)  rates 

*  Simulations  done  for  the  TPM  process  showed  that 
dissemination  of  near-line  data  (information  from  tapes  in  a 
robot)  and  off-line  data  (information  from  tapes  on  a  shelf) 
were  significant  risks 

*  The  program  continues  to  monitor  these  TPMs 

*  Modifications  to  the  system/hardware  design  and 
architecture  may  be  necessary  to  ensure  satisfaction  of  the 
near-line  and  off-line  dissemination  requirements 
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What  might  change? 

*  Only  create  TPMs  for  requirements  where  you  can 
change  something 

*  In  the  solar  oven  example  the  design  was 
changed  twice  and  the  goal  was  also  changed 

*  Obviously,  cost  and  schedule  can  be  changed  to 
improve  performance 

*  TPMs  can  be  used  to  choose  between  alternative 
concepts.  The  alternatives  that  can  be  used  to 
reduce  blood  pressure  include  drugs,  exercise, 
diet  and  reducing  alcohol  consumption.  If  one 
technique  is  not  working,  then  you  can  add  or 
switch  to  another. 
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Quantifying  System  Performance 

*  Evaluation  criteria  ( which  are  also  called  figures 
of  merit  and  measures  of  effectiveness)  are  used 
to  quantify  requirements  and  to  help  select 
amongst  alternative  designs  in  tradeoff  studies 

*  Measures  (which  used  to  be  called  metrics)  are 
used  to  help  manage  a  company's  processes 

*  Technical  performance  measures  are  used  to 
mitigate  risk  during  design  and  manufacturing 
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Examples  of  Measures,  not  TPMs 

-  Number  of  features  implemented 

-  Components  designed 

-  Components  implemented 

-  Components  integrated  and  tested 

-  Requirements  allocated 

-  Requirements  tested 

-  Test  cases  completed 

-  Paths  tested 

-  Problem  reports  resolved 

-  Reviews  completed 

-  Changes  implemented 

-  Hours  between  failures 

-  Failure  Rate  a.k.a.  Failure  Intensity 

Most  of  these  are  process,  not  product  related 


Failure  Rate 
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From  David  N.  Card,  Software  Productivity  Consortium 


In  this  case  the  planned  values  are  given  with  an  equation 

/i = vf 

where  A  is  the  failure  rate,  A0  is  the  initial  failure  rate,  6  is  the  decay 
rate  and  t  is  time.  This  is  the  equation  for  a  Poisson  distribution 


BAE  SYSTEMS 


Preventing  deterioration 

*  We  use  TPMs  for  requirements  where  the  desired 
performance  is  expect  to  improve  with  time 

*  Another  use  of  TPMs  would  be  to  prevent 
unacceptable  decreases  in  performance 

*  In  the  design  and  development  process,  adding 
bells  and  whistles  might  reduce  processing  time 
or  increase  weight 

*  TPMs  could  warn  of  such  unwanted  behavior 
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*  TPMs  are  used  to  identify  and  track  performance 
requirements  that  are  program  critical 

*  TPMs  are  used  to  establish  the  appropriate 
design  emphasis,  design  criteria  and  identify 
levels  of  technical  risk 

*  TPM  measurements 

are  collected  and 
tracked  against 
project  design 
objectives  in  the 
project’s  risk  register 
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TPM  Summary2 

Create  TPMs  for  high  priority  requirements 

-  that  impact 

•  mission  accomplishment 

•  customer  satisfaction 

•  system  usefulness 

-  where  performance  improves  with  time 

-  where  performance  can  be  controlled 

-  where  management  can  tradeoff  cost,  schedule  and 
performance 
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To  present  a  methodology  for  use  in  the  Systems  Engineering  process  that 
assists  decision  makers  in  identifying  the  unmanned  aerial  vehicle 
survivability  alternative  that  provides  the  lowest  life  cycle  cost  while  meeting 
the  operational  need. 
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•BACKGROUND 

•Systems  Engineering 
•Survivability 

•Unmanned  Aerial  Systems 

•METHODOLOGY  DESCRIPTION 
•Basic  Premise 
•Characteristics 
•Description 

•EXAMPLE 

•Scenario  Description 
•Vignette  Snapshot 
•Results 

•CONTRIBUTORS 
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BACKGROUND 

SYSTEMS  ENGINEERING 


DoD  Directive  5000.1 

Systems  Engineering.Acquisition  programs  shall  be  managed  through  the  application  of  a 
systems  engineering  approach  that  optimizes  total  system  performance  and  minimized 
total  ownership  costs. 

DoD  Instruction  5000.2 

Effective  sustainment  of  weapon  systems  begins  with  the  design  and  development  of  reliable 
and  maintainable  systems  through  the  continuous  application  of  a  robust  systems 
engineering  methodology 

Defense  Acquisition  Guidebook 

Chapter  4  describes  the  systems  engineering  processes  and  the  fundamentals  of  their 
Application  to  DOD  acquisition. 


THE  METHODOLOGY  DESCRIBED  IN  THIS  PRESENTATION  WAS  CONCEIVED 
TO  ASSIST  SURVIVABILITY  EVALUATIONS  WITHIN  THIS  PROCESS 
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BACKGROUND 

WHAT  IS  SURVIVABILITY  ? 


•  Survivability  is  the  capability  of  a  system/platform  to 
avoid  and/or  withstand  a  man-made  hostile  environment. 
Survivability  is  broken  down  into  two  subsets, 
susceptibility  and  vulnerability. 

-  Susceptibility  is  the  inability  of  an  aircraft  to  avoid  being  hit 
by  one  or  more  damage  mechanisms. 

-  Vulnerability  is  the  inability  of  an  aircraft  to  withstand  the 
damage  sustained  from  man-made  threats. 


Probability 

of 

Survival 
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BACKGROUND 

SURVIVABILITY 


•Every  combat  system  has  survivability  characteristics 

•Influenced  by  mission/threat  -  system  design/configuration  - 
relationships  to  other  systems 

•Survivability  characteristics  have  a  strong  influence  on  Total  System  Cost 
•Not  enough  survivability  -  lose  assets  and  cannot  complete  mission 
•Unnecessary  survivability  -  creates  affordability  issues 

•Survivability  is  important  to  any  warfighting  system 
It  must  survive  to  perform  the  mission 
It  protects  the  operator  from  harm 
It  keeps  the  system  affordable 


ANY  SYSTEMS  LEVEL  EVALUATION  OF  UAVs  SHOULD  INCLUDE 
A  STRUCTURED,  INTEGRATED  ASSESSMENT  OF  SURVIVABILITY 
TO  IDENTIFY  AND  DEVELOP  THE  BEST  OVERALL  CONFIGURATION 
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UNMANNED  AERIAL  SYSTEMS 

Source:  DoD  UAS  Roadmap  2005  -  2030 


A* r  j VI t\i ititi ta □' H -a rJ n Corps 


Weight 

Length: 

Wingspan: 

PfryliMid: 

Cdlijig: 

Radius: 

I  ‘  □  i  ■  J  U  i ;  i  l'i  u 


4.5  lb 
2.4  ft 
S.Sft 
L  ]h 
1000  ft 
2.i  Lkfu 
45*6 0  t'liic'i 


KQ-4  GJubdL  Hmi^Nurthru|p  GiUimraan-'AI  i-  Fitrut' 

Weight 

26,730  ll> 

Length: 

44.4  ft 

Wingspan: 

1 1 62  ft 

Rayloai 

14501b 

Ceiling. 

63,00011 

Radios: 

3400  jluj 

Endimuce: 

32  hr 

THE  TRADE  SPACE  FOR  SURVIVABILITY  IS  LARGE  AND  GROWING 
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UNMANNED  AERIAL  SYSTEMS 

Source:  DoD  UAS  Roadmap  2005  -  2030 


System 

Aircraft 

Cost,FY04$* 

Aircraft 
Weight,  Lb* 

Payload 
Capacity,  Lb 

System  Cost 
FY04$ 

Number 

Acft/System 

Dragon  Eye 

$28. 5K 

3.5 

1 

$130.3K 

3 

RQ-7A  Shadow 

$0.39M 

216 

60 

$12. 7M 

4 

RQ-2B  Pioneer 

$0.65M 

307 

75 

$17. 2M 

5 

RQ-8B  Fire  Scout 

$4.1  M 

1765 

600 

$21. 9M 

4 

RQ-5A  Hunter 

$1.2M 

1170 

200 

$26. 5M 

8 

MQ-1 B  Predator 

$2.7M 

1680 

450** 

$24. 7M 

4 

MQ-9A  Predator 

$5.2M 

3050 

750** 

$45.1  M 

4 

RQ-4(Block  10) 
Global  Hawk 

$19. 0M 

9200 

1950 

$57.7M 

1 

RQ-4(Block  10) 
Global  Hawk 

$26.5M 

15400 

3000 

$62. 2M 

1 

*  Aircraft  costs  are  minus  sensor  cost,  and  aircraft  weights  are  minus  fuel  and  payload  capacities 
**  Internal  payload  weight  capacity  only 
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BACKGROUND 


UNMANNED  AERIAL  SYSTEMS 


O 

A 


LOSSES 

LIFE  CYCLE  COST 


*  As  cost  and  military  worth  go  up,  reducing  losses  becomes  key 

*  Cost  AND  military  worth  must  be  quantified  to  support  survivability  goals 
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V 


METHODOLOGY 

BASIC  PREMISE 


TOTAL 

SYSTEM 

COST 


COST  of  SURVIVABILITY  FEATURES 
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METHODOLOGY 

CHARACTERISTICS 


•Encompass  consideration  of  all  aspects  of  survivability 

•Threat,  Mission,  Performance,  Mission  Equipment,  Survivability 
Enhancements,  Network  Functions 
•Executable  within  available  time  and  resources. 

•Account  for  cost  implications  during  normal  and  combat  conditions. 
•Methodology  supports  decision-making  even  when  little  is  known  or 
when  changes  are  encountered 

•Potential  use  as  a  capability  evaluation  tool 
•Parametric  analysis  around  inputs  that  are  “soft” 

•Analysis  allows  building  block  approach 

•Build  on  what  we  have  without  starting  over 

•Improve  fidelity  by  evolution 

•What  we  know/don’t  know  is  always  transparent 


ARRIVE  AT  THE  BEST  TOTAL  COST  ESTIMATE  POSSIBLE 
COMMENSURATE  WITH  THE  INFORMATION,  RESOURCES,  AND  TIME 

AVAILABLE. 
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METHODOLOGY 

DESCRIPTION 


R&D  Cost  Unit  Acquisition  Cost 


Hourly  Costs 


•System  Characteristics 
Platform 

Mission  Equipment 
Survivability 
•Mission  Requirements 
•Environment 
Friendly  force 
Red  Force 

•NETWORK  FACTORS 


Representative 

Vignettes 


Mission 

Models 


i 


Capability 

Achieved? 


•Losses  — 
•Hours  Flown 


CALCULATE 

TOTAL 

SYSTEM 

COST 


NO 


YES 


INTEGRATED  SURVIVABILITY  ASSESSMENT 
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EXAMPLE 

SCENARIO  DESCRIPTION 


The  sample  analysis  involves  VTUAVs  on  a  surveillance  mission 
to  locate  threat  RF  missile  sites. 

Threat  -  Three  batteries  of  short  range  RF  missiles 

Each  battery  has  three  TELARs  and  a  C2  vehicle.  Batteries 
operate  under  strict  control  of  the  commander 
One  squad  of  three  soldiers  with  each  of  the  nine  TELARs  and 
C2  vehicles  for  a  total  of  36  MANPADS.  Operate  autonomously. 

Friendly  -  Three  VTUAV  systems,  each  with  three  VTUAVs  and  a  ground 
control  station.  The  UAVs  fly  at  lOOkts  at  an  altitude  of  1050m. 


Each  has  an  EO/IR  sensor  and  an  LDRF.  When  the  ground  target 
is  detected,  the  info  is  transmitted  to  the  ground  station  which 
sends  an  attack  aircraft.  The  RF  signature  was  held  a  10  Sqm 
and  the  IR  signature  was  varied  from  500W/sr  to  1  W/sr.  A 
degrade  to  the  missile  pk  of  25%,  50%  and  75%  was  applied  to 
simulate  an  IRCM. 
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o  i 

-50 

-100 

-150 

-200 

-250 

-300 

-350 

-400 

-450 

-150  -100  -50  0  50  100  150  200  250 


Drones 


300 
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OVERALL  RESULTS 


Attrition  results  as  a  function  of  signature 


SIGNATURE 

RF  Shots 

IR  Shots 

RF  hits 

IR  hits 

500W/sr 

2.43 

17.10 

0.93 

4.33 

50W/sr 

2.80 

16.50 

0.87 

4.50 

5W/sr 

2.33 

16.87 

1.00 

4.63 

IW/sr 

1.93 

6.43 

1.00 

1.73 

Attrition  results  as  a  function  of  IRCM  effectiveness 


Pk  Degrade 

RF  Shots 

IR  Shots 

RF  hits 

IR  hits 

No  Degrade 

2.43 

17.10 

0.93 

4.33 

25% 

2.57 

18.50 

0.93 

3.87 

50% 

2.13 

19.70 

1.00 

3.30 

75% 

1.97 

21.70 

0.97 

2.00 
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BASIC 

ALQ-144 

ADV JAMMERS 

DIR  ENERGY 

Original  Number  of  Mission  Aircraft 

9 

9 

9 

9 

Number  of  Mission  Aircraft  Lost 

5.26 

4.8 

4.3 

2.97 

Mission  Probability  of  Survival 

0.41556 

0.46667 

0.52222 

0.67000 

Fleet  Size 

100 

100 

100 

100 

Number  of  Missions 

90 

90 

90 

90 

Number  of  Losses 

52.6 

48 

43 

29.7 

Number  of  Flight  Hours/Mission 

6 

6 

6 

6 

Development  Cost  ($) 

Basic  Platform 

8,000,000 

8,000,000 

8,200,000 

8,400,000 

Mission  Package 

2,000,000 

2,000,000 

2,000,000 

2,000,000 

Survivability  Enhancements 

0 

500,000 

1,000,000 

1,500,000 

Sub  Total 

10,000,000 

n 

10,500,000 

11,200,000 

11,900,000 

Unit  Acquisition  Cost  ($) 

Basic  Platform 

3,000,000 

3,000,000 

3,000,000 

3,200,000 

Mission  Package 

1,000,000 

1,000,000 

1,000,000 

1,000,000 

Survivability  Enhancements 

0 

100,000 

200,000 

500,000 

Sub-Total 

610,400,000 

606,800,000 

600,600,000 

609,590,000 

Hourlv  Operational  Cost  ($) 

Basic  Platform 

300 

300 

300 

300 

Mission  Package 

50 

50 

50 

50 

Survivability  Enhancements 

0 

10 

10 

10 

Sub  Total 

189,000 

194,400 

194,400 

194,400 

TOTAL  SYSTEM  COST  ($) 

620,589,000 

617,494,400 

611,994,400 

621,684,400 
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EFFECTS  OF  IRCM  IMPROVEMENTS 
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SIGNATURE  REDUCTION 


50  W/Sr 

5  W/Sr 

1  W/Sr 

Original  Number  of  Mission  Aircraft 

9 

9 

9 

Number  of  Mission  Aircraft  Lost 

5.37 

5.63 

2.73 

Mission  Probability  of  Survival 

0.40333 

0.37444 

0.69667 

Fleet  Size 

100 

100 

100 

Number  of  Missions 

90 

90 

90 

Number  of  Losses 

53.7 

56.3 

27.3 

Number  of  Flight  Hours/Mission 

6 

6 

6 

Development  Cost  ($) 

Basic  Platform 

8,000,000 

8,500,000 

10,000,000 

Mission  Package 

2,000,000 

2,000,000 

2,000,000 

Survivability  Enhancements 

0 

500,000 

2,500,000 

Sub  Total 

10,000,000 

11,000,000 

14,500,000 

Unit  Acquisition  Cost  ($) 

Basic  Platform 

3,000,000 

3,000,000 

3,200,000 

Mission  Package 

1,000,000 

1,000,000 

1,000,000 

Survivability  Enhancements 

0 

100,000 

200,000 

Sub-Total 

614,800,000 

640,830,000 

560,120,000 

Hourlv  Operational  Cost  ($) 

Basic  Platform 

300 

300 

300 

Mission  Package 

50 

50 

50 

Survivability  Enhancements 

0 

0 

10 

Sub  Total 

189,000 

189,000 

194,400 

TOTAL  SYSTEM  COST  ($) 

624,989,000 

652,019,000 

574,814,400 

NDIA  8th  Annual  Systems  Engineering  Conference 
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SIR/ICE  a 

ENGINEERING  COMPANY 


EFFECTS  OF  IR  SIGNATURE  REDUCTION 


60 


40 


co 

HI 

CO 

CO 
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6 


4 


50  W/Sr  5  W/Sr  1  W/Sr 


NOTE:  EXAMPLE  ONLY 
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SIR/ICE 

ENGINEERING  COMPANY 


DR.  GREG  BORN 
MISSION  MODELS 
greg.born@survice.com 


MR.  DAVE  HALL 

INTEGRATED  SURVIVABILTIY  ASSESSMENT 
dave.hall@survice.com 


DR.  JIM  WALBERT 
NETWORK  ANALYSIS 
jim.walbert@survice.com 


NDIA  8th  Annual  Systems  Engineering  Conference 
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SIR/ICE 

ENGINEERING  COMPANY 


QUESTIONS? 
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NDIA  8th  Annual  Systems  Engineering 

Conference 

" Requirements  Management  Tips 

and  Tricks ” 


October  27,  2005 


Frank  Salvatore 

High  Performance  Technologies,  inc. 

3159  Schrader  Road 

Dover  NJ,  07801 

(973)  442-6436  ext  249 

fsalvatore@hpti.com 


Outline 


□  Requirements  Elicitation 

□  Requirements  Capture  and  Management 

□  Requirements  Traceability 

□  Requirements  Control 

□  Reaching  Consensus 

□  Eliciting  Verifications 

□  Communicating  Requirements 

□  Metrics 


Requirements  Elicitation 


How  do  you  gather  the  requirements? 

□  Interviews 

□  QFD  Workshops 

□  Web  Based  Surveys 

□  Vignettes  and  Scenarios 

□  Questionnaires 

□  Brainstorming  and  Mind  Mapping 

□  Analysis/Derivation 

V  Hazard 

V  Fault  Tree 

V  Sensitivity 

V  Trade  Studies 

□  Existing  Documentation  and  or  Policies 

□  Quality  Assurance  Provisions 

Don’t  forget  to  Document  Rational .  It  will  save  you  time 
latter  when  you  will  need  to  defend  the  requirements . 


It  involves  a  lot  of 
research  and  is 
evolutionary! 


Interview  Based  Elicitation 


Using  and  Enterprise  Architecture  approach  one  can  first 
probe  into  Business  Goals  and  Architecture  Principles  buy 
asking  questions  to  understand: 


□  Mission  and  Values  of  your  organization 

□  Understand  importance  (PM  Level) 

□  Understand  organization  structure 

□  Understand  Products 

□  Understand  Customers  and  Stakeholders 

□  Understand  Daily  Activities 


Mostly  used  for  Business  Systems 


Interview  Based  Elicitation 


Project  and  Product  Data  can  be  understood  by 
asking  these  leading  questions 


□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 


What  are  the  Projects/Products  that  PM 
Mortars  manages? 

Who  do  you  interact  with? 

What  data  types  do  you  manage? 

How  do  you  organize  your  data? 


What  data  do  you  view  as  being  most 
important? 


Who  are  the  Customers  for  each  product? 
Who  are  the  stakeholders  for  each  product? 

What  are  the  day  to  day  activities  that  go 
on  for  the  projects  you  choose? 


QFD  Based  Elicitation 
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43: 

153: 

203: 

153: 

73: 

23: 

153: 

Also  helps  to  Build 
Consensus  and 
Understanding  of 
complex 
relationships  as 
well  as 
importance. 
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Requirements  are  Discovered  Thru 
_ The  SW  Safety  Process 

I  SOFTWARE  SAFETY  PROCESS  I 


Eliciting  Verification  Methods 


Similar  to  Requirements.  Stakeholders  are 
different.  Methods  are  typically  thru 
Analysis,  Test,  Inspection,  Measurement. 

□  Use  Interview 

□  Use  Questionnaires 

□  Include  Stakeholders  Early  and  Often. 

□  Have  Stakeholders  Peer  Review  Requirements 

□  Use  a  JCCB 


=ile  Edit 

Insert  Records  Window  Help 

|=Jj9J 

igraph: 

Interface  Definition 

TRL: 

r  TRL5 

IPT  Name:  Ammo  Handling 

:ion: 

3.1. 2.0-1 

r  TRL6 
r  TRL7 

Module  Name  AHR 

uirements: 

The  Ammo  Handling  Subsystem  will  interface  with  the  Turret  Structure,  Gun 
Assembly,  Fire  Control,  Ammo  Suite  and  Secondary  Armament. 

ATD/Objective  Force: 

This  enables  and  disables 
the  required  field  warning: 

Switch  to  View  Mode 

RL  5  Verification  Method: 


Responsibility:  Analysis 

Inspection 

Location  of  Verif:  M  easurement 

Test 

Verification  Procedure:  Briefly  N/A 
If  the  requirement  will  not  be 


(e.g.:  IPT  Name,  Subcontractor,  System  Integrartor)  Critical  T est: 


(e.g.:  Picatinny,  Contractor  Facility,  Proving  Ground) 
d  at  this  TRL  level  to  validate  or  confirm  the  requirement. 


"Please  Select  a  Test 


Data  Collected: 


L  G  Verification  Method: 


"Please  Select  a  Method" 


Responsibility: 
Location  of  Verif: 


(e.g.:  IPT  Name,  Subcontractor,  System  Integrartor)  Critical  T est: 


(e.g.:  Picatinny,  Contractor  Facility,  Proving  Ground) 


Data  Collected: 


i  —  afeKKrjBrafawgsifliaitfTMftB 


™Please  Select  a  T est 


Verification  Procedure:  Briefly  describe  the  procedure  you  recommend  at  this  TRL  level  to  validate  or  confirm  the  requirement. 
If  the  requirement  will  not  be  verified  at  this  time  please  indicate  so: 


Record 


E 


Requirements  Capture  and 

Management 


How  and  where  do  you  store  the  requirements? 

Word  Documents  are  standard.  Tools  are  useful  and  can 
Help.  But  try  to  get  everyone  to  use  them 
consistently!!!!! 

□  Access 

□  Excel 

□  DOORS 

□  RTM 

□  Requisite  Pro 

□  RM  Calibre 

□  etc.... 


Use  Document  Templates  Based  On 
Standards.  Also  IM  is  Important  for  Efficiency. 


Requirements  Management 
Specification  Hierarchy 


Caliber 
Trade  Study 


Ammo 

Verification 


Turret 

Verification 


ATD 

Exit  Criteria 


System 

Verification 


Ammunition 

Suite 

Reqts 


/ 


System 

Requirements 


System 

Level 


FCS 

Mission  Needs 
Statement 


User 

Documents 


Main  Arm. 
Verification 


Main  Armament 
Reqts 


Sec  Arm. 
Verification 

Secondary 
Armament 
Reqts 

Product 
Level 


Turret 

Reqts 


Auto  Loader 
Verification 

Autoloader 

Reqts 


Gun  Assembly 
Verification 


Gun  Assembly 
Reqts 


Fire  Control 
Verification 


Fire  Control 
Reqts 


Sub-System 

Level 


Gun  Mount 
Verification 


Gun  Mount 
Requirements 


Launcher 

Verification 


Launcher 

Requirements 


Assembly 

Level 


Follows  IEEE  Commercial  Standards 


Configuration 

Item 


Recoil 

Mechanism 

Reqts 


Configuration 
Item 


Cradle  Assy 
Reqts 


Component 

Level 


Document  Outline  is  Standard 

Throughout  Project. 


Formal  module  712  Sys/Sys  Reqts1  current  4.0  (RFP  12_6_01)  -  DOORS 


File  Edit  View  Insert  Link  Analysis  Table  Tools  User  Help 


JfllxJ 


X  v'  r  m*  b  i  u 


|  [standard  ■ 


|  Level  2 


\ 

□□□ 


*£  S  3  s  s  Tt  H 


ID 


MRAAS  System  Requirements 


SYSR37 

SYSR38 

SYSR39 

SYSR40 

SYSR41 

SYSR42 


1  SCOPE 

2  APPLICABLE  DOCUMENTS 

>2.1  Government  Documents 
2.2  Non-Government  Document 

3  REQUIREMENTS 

>  3.1  MRAAS  System  Definition 


SYSR48 


SYSR55 

SYSR63 

SYSR64 

SYSR68 

SYSR71 

SYSR72 

SYSR73 

SYSR78 

SYSR79 

SYSR80 

SYSR81 

SYSR82 

SYSR83 


>  3.2  Characteristics 


>  3.3  Design  and  Construction 
3.4  Documentation 

>  3.5  Logistics 

>  3.6  Personnel  and  Training 

3.7  Major  Component  Characteristics 

3.8  Precedence 

4  QUALITY  ASSURANCE 

5  PREPARATION  FOR  DELIVERY  N/A 

6  NOTES 

7  SCHEDULE 

8  TECHNOLOGIES  TO  INVESTIGATE 

9  This  section  intentionally  left  blank 

10  APPENDIX 


jd 


Username:  fsalvatore 


Exclusive  edit  mode 


.lT1 


/a 


HUsing  Mil-STD-490 
standard  template 

^Standardized 
Documentation  format 
makes  it  easier  to  find 
what  you  are  looking 
for 


Level  1  User  Requirements 


Level  2  System  Requirements 


Level  3  Product  Requirements 


Level  4-6  Subassembly  to 
Component  Requirements 


Requirements  Traceability 


How  do  you  understand  how  the  requirements 
are  being  satisfied,  are  complete,  are 
accurate,  etc . 

□  Trace  Matrices  are  Typical  and  require  constant  care  and 
feeding  to  maintain. 

□  Use  a  tool  to  manage  your  requirements  and 
capture  traceability  so  you  can  search  and  query 
when  doing  impact  analysis. 


S  More  accurate 
S  More  efficient 
S  More  complete 


If  a  requirement  isn’t  traceable 
to  anything  it  doesn’t  belong!!! 


No  tool  will  automatically 
generate  but  they  will 
preserve  it  once  you  do  it  the 
first  time . 


This  is  Important  when 
performing  Impact  Analysis , 
doing  FCA  and  PCA,  etc .... 


Requirements  Change  Control 


If  a  Requirement  is  changed,  how  do  we  determine 
effects  on  other  Requirements,  Verifications  or 
Schedule  Events? 

□  Use  Inter-IPT  Coordination 

□  Use  Impact  Analysis  &  Visualization  Tools 

□  Use  Formal  Change  Control  Procedures 

□  Attributes 


With  a  tool  you  have  better  and  more  efficient  ways  of 

controlling  the  requirements. 


Follow  a  Change  Proposal 

Process 


Starting  the  Change  Process 


IPT  Member  brings  an  issue  to  attention  of  IPT  Lead 

IPT  Lead  makes  an  initial  determination: 

PURSUE  -  Proposed  change  has  merit  and  is  worth  further 
investigation 

DISCARD  -  Proposed  change  does  not  have  merit  or  is  not 
worth  further  investigation  at  this  time 

If  you  choose  to  PURSUE  the  potential  change: 

1.  Coordinate  with  other  IPT's  to  discuss 

2.  Initiate  working  group(s)  as  needed 


COMMUNICATE  !!! 


Starting  the  Change  Process 


Still  think  a  change  is  needed?  Perform  an 
“Impact  Analysis” 


Step  2 

SibmrtClaige 
Proposal  aid  or 
Suggestions. 
Sibmitadditbial 
CPs  brinpacfcd 
objects. 


Step  3 


Reuiew  CPs  aid 
Siggestbis  br 
Submittal  to  CCB. 


Step  4  &  6 


Detimiie  wlblCPsaid 
Siggestbisbreuiewaid 
asemble  neuiew  package/ 
CP  Let  DEtrbife  actbis. 


Step  5 


CoidictCCB  ReuiovoS 
Dspositbi  of  CP  aid 
Siggestbis 


coltt»rate  “■ 


Reuiew  Package 


Prd 

Mer 

Uect 

nber 

IPT 

Rep 

O 

IPT 

ther 

Reps 

Cl 

Ul 

-Reject 


-Problem  Defectd 


Reqiinemeife  Dative 


Step? 


Coordiiate  brmal 
claige  actbis  b 
tie  neqiinemeifc 
database. 


Accept- 


■ 

START 

1 

— 

- ■ 

FINISH 

Impact  Analysis  Complete ... 
Submit  a  Change  Proposal 


Prdject 

Member 


Rep 

O 

IPT 

ther 

Reps 

C 

11 

Reject- 


-Pnotilem  Defectd 


■ 

START 

l 

FINISH 

-Wf 


Reqiiremeite  Dattse 


Submit  Change  Proposal 


Fill  out  appropriate  fields  in  the  ‘Proposed’  half  of  the  Change  proposal  Form.  Remember 
to  address  any  affected  attributes. 


|  Change  Proposal  for  module  'LAR'  -  DOORS 


Change  proposal  for  object:  LAR360 
Pending  change  proposals  for  this  object:  1 
Current 


El 

In-links:  0 
□  ut-links:  1 


Object  Heading 


Object  T ext 


The  muzzle  brake  shall  not  generate  a  muzzle  exit  -J 

pressure  above  12ksi. 


|atd 


31 


Proposed 
Object  Heading 


Object  Text 


The  muzzle  brake  shall  not  generate  a  muzzle  blast 
overpressure  above  TBD.  (Driven  by  muzzle  exit 
pressure  of  12  ksii - 1 - 5 - 


Show 


pTD 


Make  adjustments  to  the 
attribute:  JaTDA  Reason  for  change  as  needed. 

BE  SURE  TO  NOTATE  ANY 
CONTRACTUAL 
IMPLICATIONS!!! 


Reason  for  change: 


uzzle  blast  overpressure  is  correct  term.  Muzzle  Brake  will  be  designed  to  minimize  blast  overpressure. 
Other  impactSfrrequirements  are: 


Change  type:  |  Modify  this  object  Priority: Medium 


3 


Submit 


Select  Very  High,  h,  I  or 

(refer  to  CPP  Document  for  details) 


4- 


When  satisfied  with 
form,  press  Submit  to 
create  the  new  Change 
proposal 


Submit  Change  Suggestion 


When  5  or  more  actions  need  to  occur  (I.e.,  Change  proposals)  in  order  to  fully  satisfy  a 
Change  Proposal,  a  Change  Suggestion  should  be  created  instead  of  a  change  proposal. 


Fill  out  fields  as  needed  and  press  Submit  to  create  a  new  suggestion.  The  JCCB  will 
approve  and  apply  suggestions  via  the  Change  Proposal  System. 


Review  CP’s  and  Suggestion 


Project 

Member 


Rep 

O 

IPT 

ther 

Reps 

C 

11 

-  Reject - 


-Pnotitem  Detctd 


■ 

START 

l 

FINISH 

-Wf 


Reqiiremeife  Dative 


Predefined  Views  Can  Help 


Forms  Can  Also  Help 


ID  CP’s  and  Suggestions  and 

Schedule  JCCB 


Project 

Member 


Rep 

O 

IPT 

ther 

Reps 

C 

11 

-  Reject - 


-Pnotitem  Detctd 


■ 

START 

l 

FINISH 

-Wf 


Reqiiremeife  Dattrae 


Perform  JCCB  and  Update  dB  with 

Results . 


Step  1 


Perform  Impact 
Aialysteaid 
cultimrat  witi 
IPT  Rep  to  create 

CP(s). 


Step  2 

SibmrtClaige 
Proposal  aid  or 
Suggestions. 
Sibmitadditbial 
CPsbrinpacted 
objects. 


Step  3 


Reuiew  CPs  aid 
Siggestbis  br 
Submittal  to  CCB. 


Step  4  &  6 


Deteimiie  wlblCPsaid 
Siggestbis  breuiew  aid 
ssemhte  reuiew  package/ 
CP  Let  DEtrtufcactbis. 


L-  col  Isolate 


Coordiiate  brmal 
claige  actbis  b 
tie  reqiiremeite 
database. 


Ii-Reuiew 


Reuiew  Package 


Prd 

Met 

Uect 

nber 

IPT 

Rep 

O 

IPT 

ther 

Reps 

Cl 

U1 

-m-  Accept - 


CCE 


-Reject- 


-Prctilem  Defected 


■ 

START 

1 

— 

- ■ 

FINISH 

Reqiiremeite  DaHrae 


Approved  (ready  for  implementation) 
On-Hold  (further  investigation  needed) 
Rejected  (requested  change  discarded) 


Reaching  Consensus 


Use  IPT  forum  to  Elicit  Requirements. 

□  Include  Stakeholders  Early  and  Often. 

□  Have  Stakeholders  Peer  Review  Requirements 

□  Document  Rational.  It  will  save  you  time  latter 
when  you  will  need  to  defend  the  requirements. 

□  Use  a  JCCB 

□  Try  using  QFD  Method  to  Build  Consensus 


Communicating  Requirements 


Use  of  DOORS  has  helped  BUT!! 

□  Culture  shock  is  hard  to  overcome. 

□  Revert  back  to  WORD  and  EXCEL  documents. 

Not  so  efficient  and  may  introduce  errors. 

□  May  need  to  hold  hands 

□  Provide  Training  and  Tailor  it  to  the  project. 

□  Need  to  pay  close  attention  to  Permission  and 
database  administration  details. 

□  JCCB  has  forced  communication  to  happen  and 
has  made  it  mandatory. 

□  Will  need  good  IT  support  to  reach  remote 
locations  when  using  a  tool. 


Requirements  Metrics 


Select  metrics  you  will  use. 

Don’t  try  to  many  or  they  won’t  be  managed. 
You  can  build  them  into  an  RM  tool. 


Some  Examples  Include: 
Volatility 

#  Requirements 

#  TBD 


#  Verified 


Using  a  tool  will  produce 
metrics  naturally. 


Requirements  Attributes 


Attributes  are  additional  defined  characteristics 
of  a  requirement  and  they  provide  essential 
information  in  addition  to  requirement  text 


Source 

Priority 

Verifiability 

Accepted 

Review 

Safety 

Comments 

Questions 


Who  specified  this  requirement? 

What  is  the  priority  of  this  requirement? 

Is  the  requirement  verifiable? 

Has  this  requirement  been  accepted  by  the  developers? 
Review  status  of  this  requirement 
Is  this  a  safety-critical  requirement? 

Any  comments  on  the  requirement  to  clarify  its  meaning 
Any  questions  that  must  be  clarified  with  the  source 


You  can  define  attributes  that  will  support  your 
process  and  make  your  database  more 
productive  for  you 


Summary 


The  use  of  an  RM  tool  is  an  enabling  technology  to  achieve 
greater  accuracy  and  efficiency  when  engineering 
requirements. 

There  are  definite  skills  and  disciplines  required  to  do 
requirements  engineering 

Not  only  will  One  need  to  understand  how  to: 

□  Elicit  Requirements 

□  Capture  and  Control  Them 

□  Establish  and  maintain  Traceability 

□  Reach  Consensus 

□  Elicit  Verification  Methods 

□  Communicate  Requirements 

□  Defined  some  Metrics  and  Attributes 

They  will  also  need  to  be  proficient  in  using  and  tailoring  an  RM 
Tool 


Questions? 


•  Chaos  Theory  is  the  name  science  has  come  up  with  to  describe  the  very 
complex  way  the  world  works. 

-  Much  of  mathematics  is  "linear",  or  related  to  a  line,  making  equations  and  figuring 
out  the  answer  fairly  straight  forward. 

•  But  there  are  some  things  that  just  can’t  be  explained  so  easily,  like  weather 
patterns,  ocean  currents,  and  defense  logistics.  There  are  too  many  things 
going  on  to  keep  track  of:  It  almost  seems  as  if  they  are  random,  or  "chaotic". 

-  Chaos  theory  is  a  way  describe  and  predict  these  types  of  events. 

•  As  a  Chaos  Theory,  defense  logistics  process  streamlining  is  next  to 
impossible  without  reference  modeling,  as  End-to-End  Logistics  spans  the 
Galaxy! 

-  Reference  models  visualize  the  “Best  of  Breed”  across  the  National  Technology 
Industrial  Base 

•  Reference  Models  feed  off  of  logistics  data:  better  data,  better  results 

•  As  a  Chaos  Theory,  defense  logistics  data  analysis  requires  a  common 
logistics  data  schema,  as  data  files  are  so  huge  and  tedious. 

-  A  common  data  schema  is  tantamount  to  logistics  data  linkage 

- - ■ 
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•  Operations  Reference  Models  -  what  are  they? 

•  A  Perspective  On  Life  Cycle  Logistics 

•  What  is  Industry  Using  for  operations  modeling? 

-  Supply  Chain  Operations  Reference  model 

-  Design  Chain  Operations  Reference  model 

•  The  Need  for  Information 

-  Common  Logistics  Data  Schema 

•  Bringing  it  All  Together  (a  Notional  Concept) 

•  A  parting  Shot 

■  - ==^ -< 
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•  Process  reference  models  integrate  the  well-known  concepts  of 
business  process  reengineering,  benchmarking,  and  process 
measurement  into  a  cross-functional  framework 


Business  Process 
Reengineering 


Benchmarking 


Best  Practices 
Analysis 


Capture  the  “as-is” 
state  of  a  process 
and  derive  the 
desired  “to-be” 
future  state 


0. 


Quantify  the 
operational 
performance  of 
similar  companies 
and  establish 
internal  targets 
based  on  “best-in¬ 
class”  results 


k 


Characterize  the 
management 
practices  and 
software  solutions 
that  result  in  “best- 
in-class” 
performance 


0 

I 

m 

0 


Process  Reference 
Model 

Capture  the  “as-is”  state 
of  a  process  and  derive 
the  desired  “to-be”  future 
state 


Quantify  the  operational 
performance  of  similar 
companies  and  establish 
internal  targets  based  on 
“best-in-class”  results 

Characterize  the 
management 
practices  and 
software  solutions 
that  result  in  “best-in¬ 
class”  performance 


Dafd  is  the  fuel  for  reference  models 


Supply>CkAiN  CouNcil 


Program  $$$ 


icd  Two  Open  Chains... without  adequate  linkage! 


Pre-Systems  Acquisition 


Systems  Acquisition 


Design  Chain 


from  Raw  materials  to  End  User, 
i.e.,  systems  fielding  plan 


Sustainment 

Supply  Chain 


From  End  user  to  Recovery 
to  Multiple  End  users 
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m 
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The  Supply-Chain  Operations  Reference-model  (SCOR) 


>SCOR  is  a  management  tool  that  has  been  developed  by  the 
Supply-Chain  Council  as  the  standard  diagnostic  tool  for  supply-chain 
management,  enabling  users  to  address,  improve,  and  communicate 
supply-chain  management  practices. 

>The  SCOR-model: 

> Describes  the  business  activities  associated  with  all  phases  of 
satisfying  a  demand. 

>  Utilizes  process  building  blocks. 

> Identifies  metrics. 

>  Uses  a  common  set  of  definitions. 

>  Links  virtually  any  supply  chain  within  Government  and  Industry. 


Supply^CkAiiN  Council 

Suppliers 


Master  Scheduling  to  Meet  the  Warfighter’s  Needs! 


Plan 


P2  Plan  Source 


PI  Plan  Supply  Chain 


P3  Plan  Make 


P4  Plan  Deliver  P5  Plan  Returns 


SRI:  Return 
Source 


DR1:  Return 
Deliver 


EP1:  Enable 
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CD 

E 
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Supply^CkAiiN  Council 


P2:  SOURCE  P3:  MAKE  P4:  DELIVER  P5:  RETURN 


Ejl  SI :  Source  Stocked 
:=====  Product 

Best  Practice: 

53  Joint  Service 
:}ij  Agreements 


S2:  Source  Make-to- 
order  Product 
Best  Practice: 
Statistical  Process 
Control 


Ml:  Make-to-Stock 
Best  Practice: 
Benchmarking  Six  Sigma 


H  Dl:  Deliver  Stocked 
Product 
Best  practice: 
pi  Electronic  Catalogs 
Quick  Response 


M2:  Make-to-order 
Best  Practice:  Capacity 
Planning 


pH  D2:  Deliver  Make-to- 
order  Product 
h|  Metrics: 

BB  Fill  Rates 


S3:  Source  Engineer- 
to-order  Metrics: 
Product  Acquisition 
Costs 


M3:  Engineer-to-Order 
Best  Practice: 
Demand-pull 
manufacturing 


M  b3:  Deliver  Engineer- 
to-order  product 
111  Metrics:  Order 
US  Management 


SRI :  Source  return  defective 
product  Metrics:  Cycle  time 


DR1:  Deliver  return  defective 
product  Metrics:  Cycle  time 


EP2:  Manage  Performance  of  Supply  Chain  (MACOM) 


CUSTOMERS 


Suppliers 


System  Design  for  Operational  Effectiveness 


Product 

Support 

Provider 


Product  I 

Support  | 

Integrator  j 


MACOM 


Warfighter 


Program  Manager 


Common  Logistics  Data  Schema 


Performance  Based  Logistics  Approach  j 

PBL  Contract  Metrics  S 


Technology 


Life  Cycle  Logistics 
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Example  -  the  Government  Electronics  and  Information  Post-Fieidjng  supp°rt 


Technology  Association  (GEIA)  standard  927  - 

Multiview 


Multiview  -  an  integrated 
multi-domain  data  schema  ^ 
for  representing  system 
product  and  process  data. 


Excel 


SPAR 
(Simulation) 


Database  Interface  Mechanisms 


The  data  schema  is  the  organization 
and  interrelationships  of  system  data 
essential  for  developing  an  advanced 

integrated  environment 


USAMC  LOGSA-Supporting  Warfighters  Globall 


Program  $$$ 


Pre-Systems  Acquisition 


Systems  Acquisition 


Sustainment 


Dayl 


Total  Life  Cycle  Systems  Management 


System  Design  for 
Operational  Effectiveness 


Performance  Based  Logistics 
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•  A  Milestone  “D”,  with  exit  criteria  and  a  Sustaining  Performance  Document  (notional)  could  be  the  conduit 
between  Acquisition  and  Sustainment. 

-  Presently,  the  biggest  life  cycle  event  has  no  criteria 

•  Cost,  Schedule,  Performance,  &  Supportability  under  one  focal  point  across  the  Life  Cycle 

•  Sustainment  currently  relies  too  heavily  on  forensics  to  determine  plan  of  action 

-  Need  to  map  the  requirements  from  Technology  Development  to  operations  &  support 

•  Move  beyond  “respond  and  fix” 

-  Needs  to  become  a  value  added  service 

•  Presently  “Data  Rich  and  Information  Poor” 

-  A  Common  Data  Schema  would  interact  all  facets  of  logistics  and  engineering 

•  The  “tie  that  binds”  between  engineers  and  logisticians! 


For  further  information  and  discussion: 

nloBYHANNA 

Uarmy  depot  "SSS— 


John  Sells 

570-895-7585 

John.sells@us.army.mil 


•Louis  A.  Kratz,  Assistant  Deputy  Under  Secretary  of 
Defense  (Logistics  Plans  and  Programs) 

•Edward  T.  Bair,  Program  Executive  Officer, 
Intelligence,  Electronic  Warfare  &  Sensors 

•Randy  Fowler,  Director,  Center  for  Logistics  and 
Sustainment  Curriculum  Development,  Defense 
Acquisition  University 

•Jerry  Cothran,  Program  Director,  Performance  Based 
Logistics,  Defense  Acquisition  University 

•Jerry  Beck,  Senior  Program  Analyst,  Office  of  the 
Assistant  Deputy  Under  Secretary  of  Defense 
(Logistics  Plans  &  Programs) 

•Joe  Burak,  Senior  Supply  Chain  Analyst,  Chairman  - 
Supply  Chain  Council,  Aerospace  &  Defense  Special 
Interest  Group 

•Veronica  Allen,  Associate  Director,  operations 
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Systems  Modeling  Language  (SysML) 

Overview  &  Update 

NDIA  Systems  Engineering  Conference 

October  27,  2004 

Rick  Steiner 

SysML  Submission  Team 
Raytheon 
(858)  522-2008 
fsteiner@raytheon.  com 


Caveat 


•  Current  baseline  for  SysML  is  v0.9  submitted  to  OMG  in 
January  05 

*  SysML  Submission  Team  and  SysML  Partners  are  two 
competing  teams  working  to  finalize  the  specification  and 
submit  for  adoption  to  the  OMG  in  February  2006 


t 


This  material  is  based  on  current  status  of  the  SysML 
Submission  Team 


A/eec^fof'^s/l^-lllllllllllllll^ . 

•  Systems  Engineers  need  a  robust  language  for  analyzing,  specifying, 
designing,  verifying  and  validating  systems 

•  Many  different  modeling  techniques 

-  Behavior  diagrams,  IDEFO,  N2  charts, ... 

General  purpose  language 

-  satisfy  broad  set  of  modeling  requirements  integrate  with  other 
disciplines  (SW,  HW, ..) 

-  be  scalable,  adaptable  to  different  SE  domains,  supported  by  multiple 
tools 

-  A  Systems  Engineering  Modeling  Language  based  on 
UML  2  has  a  good  chance  of  meeting  rnese  objectives! 

•  Joint  INCOSE  /  Object  Management  Group  (OMG)  Initiative  to  extend  UML 
to  SE 

-  Systems  Engineering  Domain  Special  Interest  Group  (SE  DSIG) 
kickoff  in  Sept  ‘01 

•  Aligned  with  ISO  AP-233  Systems  Engineering  data  interchange 
standard  to  support  tool  interoperability 

-  UML  for  SERFI  issued  in  2002 

-  UML  for  SE  RFP  (ad/03-03-41)  issued  March  28, 2003 


Structure  in  UML  2- A  Useful  Concept  for  Systems  Engineers 


Definition 

Use 

(Class  Diagram) 

(Composite  Structure  Diagram) 

SysML  Submission  Status 

•  SysML  Partners  formed  in  March,  2003 

-  SysML  V0.9  submitted  to  OMG  on  Jan  10, 2005 

•  Profiles  chapter  addendum  submitted  May  30 

-  4  tool  vendors  piloted  use  of  SysML  0.9  in  their  tools,  and 
presented  at  INCOSE  2005  symposium  in  Rochester 

•  Artisan,  EmbeddedPlus,  iLogix,  and  Telelogic 

-  Missed  goal  for  revised  submission  update  in  May  and  August  ’05 

•  SysML  Submission  Team  announced  split  from  SysML  Partners 
on  August  30, 2005  to  finalize  spec 

-  Goal  to  submit  Final  Revised  Submission  for  presentation  at 
December  ‘05  OMG  meeting 

-  Request  vote  to  recommend  adoption  at  February  ‘05  OMG  meeting 

•  SysML  1.0  should  be  ready  for  use  early  in  2006 

-  Already  appearing  in  tools  (0.9x  version) 


SysML  Diagram  Taxonomy 


Hybrid  SUV  Example  -  Context  Diagram 


Hybrid  SUV  Example  -  Requirements  Hierarchy 


req:HSUV_Requirement_Hierarchy 


Hybrid  SUV  -  Requirements  Derivation 


req:HSUV_Requirement_Derivation 


«requirement» 

«requirement» 

Braking 

FuelEconomy 

A 

1 

J 

1 

i  / 

«derive»  «derive» 


«requirement» 

RegenerativeBraking 


«derive»  «derive» 


«derive» 


«derive»  «derive»  «derive» 
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Hybrid  SUV  -  Satisfy/Verify  Requirements 


Hybrid  SUV  -  black  box  Sequence  Diagram 


Hybrid  SUV  -  Top  Level  State  Machine 


Hybrid  SUV  -  Power  System  Internal  Block  Diagram 


Hybrid  SUV-  Fuel  Economy  Equation  Constraint  Diagram 


Hybrid  SUV-  Vehicle  Dynamics  Constraint  Diagram 


Hybrid  SUV  -  Acceleration  Timing  Diagram 


Hybrid  SUV  -  Acceleration  Activity  Diagram  (EFFBDm  1) 


«EFFBD» 

act:Accelleration  Overviewb 


1 


InitializeVehicle 
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PushAccelerator 
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MeasureVehicle 
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Hybrid  SUV  -  Acceleration  Activity  Diagram  (EFFBDm  2) 


Hybrid  SUV  -  Acceleration  Activity  Diagram  (Allocation) 


Hybrid  SUV  -  Internal  Block  Diagram  with  Allocation 


ibd: Power  Subsystem  Allocated/ 


«allocated» 
{allocated  From = 
driveCurrent} 
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SysML  Submission  Team 

•  Members 

-  Industry  &  Government 

•  American  Systems,  BAE  SYSTEMS,  Boeing,  Lockheed  Martin,  Nisf, 
oose.de,  Raytheon,  THALES,  Eurostep,  EADS  Astrium 

-  Vendors 

•  Artisan,  EmbeddedPlus,  IBM,  l-Logix,  Mentor  Graphics,  Sparx  Systems 

•  Collaborations 

-  Deere  &  Company 

-  Georgia  Institute  of  technology 

-  INCOSE,  AP-233 


SysML  Milestones 


.  UML  for  SE  RFP  issued  -  March  28, 2003 

•  Kickoff  meeting  -  May  6, 2003 

•  Overview  presentation  to  OMG  ADTF  -  Oct  27, 2003 

•  Initial  draft  submitted  to  OMG  -  Jan  12, 2004 

•  INCOSE  Review  -  January  25-26, 2004 

•  INCOSE  Review  -  May  25, 2004 

•  Revised  draft  submitted  to  OMG  -  Aug  2 

•  2nd  Revised  submission  to  OMG  -  October  1 1 

•  OMG  technology  adoption  -  Q1  2005  (Goal) 


•  Structure 

-  e.g.,  system  hierarchy,  interconnection 

•  Behavior 

-  e.g.,  function-based  behavior,  state-based  behavior 

•  Properties 

-  e.g.,  parametric  models,  time  property 

•  Requirements 

-  e.g.,  requirements  hierarchy,  traceability 

•  Verification 

-  e.g.,  test  cases,  verification  results 

•  Other 

-  e.g.,  trade  studies 


4  Pillars  of  SysML 

Structure 


ABS  System: Assembly  Diagram 


Requirements 


Apply  Brakes:  Activity  Diagram/ 


Behavior 


Braking  Perfdm^cne: Parametric  Diagram/ 
^ - 


«parametricRelation» 
Total  Force  =  Sum  Forces 


«property» 

Stopping. 

distance 


«parametricRelation» 
Force  =  m*a 


«parametricRelation» 
Integrate 


«property» 

«property» 

«property» 

«property» 

Tire.friction 

Braking.friction 

Vehicle.weight 

Vehicle.speed 

«property» 
Vehicle,  dec¬ 
eleration 


Constraints 


SysML  ( AP-233  Alignment 
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*  The  Big  Question 

-  System  Safety 

-  Systems  Engineering 

*  Classic  System  Safety  Model 

*  OSD(AT&L)  Life  Cycle  Management  Framework 

*  Systems  Engineering  V-model 

*  “Integrated”  System  Safety  Model 

*  Summary 


NDIA  8th  Annual  Systems  Engineering  Conference 
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SIR/ICE 

ENGINEERING  COMPANY 


*  Have  you  ever  wondered: 

-  Why  is  it  that  it’s  Systems  Engineering, 

-  But  it’s  System  Safety? 

-  What  happened  to  the  “s”? 

-  Have  you  asked  yourself  this  same  question? 

-  And,  it’s  been  used  inconsistently  at  this  conference!! 

*  Let’s  explore  this  for  a  few  minutes 


NDIA  8th  Annual  Systems  Engineering  Conference 
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What  is  System  Safety? 


*  Engineering  of  Safe  Systems  or  Safety  of 
Systems 

*  Systems  Safety  -  the  discipline 

*  System  Safety  -  the  application  of  the  discipline 
of  systems  safety  to  a  specific  system  or  a 
system  of  systems 

*  and... 
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What  is  Systems  Engineering? 


*  Engineering  of  Systems 

*  Systems  Engineering  -  the  discipline 

*  System  Engineering  -  the  application  of  the 
discipline  of  systems  engineering  to  a  specific 
system  or  a  system  of  systems 

*  One  Air  Force  Program  Office  used  the 
terminology  Director  of  “System  Engineering” 
because  according  to  the  Director,  they  were 
working  on  only  one  system  (contextually-based) 

*  But  what  it  points  to... 


NDIA  8th  Annual  Systems  Engineering  Conference 
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r  System  Safety  versus  Systems 

Engineering 


*  Lack  of  effective  integration  of  Systems  Safety 
within  Systems  Engineering  (or  System  Safety 
within  System  Engineering  at  the  project  level) 

*  Real  issue  is  System  Safety  Requirements  and 
ensuring  System  Safety  is  effectively  integrated 
into  product  realization 

*  So. .  .what  do  we  do? 

*  First,  we  might  use  a  standard  definition  of  system 

*  But  keep  that  question  in  mind  while  we  discuss 
some  other  ideas 


NDIA  8th  Annual  Systems  Engineering  Conference 
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Classic  System  Safety  Model 

(MIL-STD-882D) 
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Classic  System  Safety  Model 

(MIL-STD-882D) 


What 
happens 
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these  two 
blocks? 
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DoD  5000.1  Acquisition  Phases 


*  Major  System  Acquisition  Phases 

-  Concept  Refinement 

-  Technology  Development 

-  System  Development  &  Demonstration 

•  System  Integration 

•  System  Demonstration 

-  Production  &  Deployment 

•  Low-rate  Initial  Production 

-  Operations  &  Support 

•  Full-Rate  Production  and  Deployment 

•  Sustainment 

•  Disposal  (Recycle/Reuse,  Reprocessing  or  Disposal) 
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DoD  5000.1  Acquisition  Phases 
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Integrated  Systems  Engineering 

“The  Wall  Chart” 


Integrated  Defense  Acquisition,  Technologic  &  Logistics  Life  Cycle  Management  Framework 
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Phase  Characteristics 


Phase-specific  Technical  Baseline 

Phase-specific  “Requirements”  Review  including 
“Derived”  Requirements 

Requirements  Analysis 

Functional  Decomposition 

Functional  and  Physical  Allocations 

Subsystem  and  Component  Specifications 

Component,  Subsystem  &  System  Integration 

Verification  and  Validation  Activities 
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SIR/ICE 

ENGINEERING  COMPANY 


Systems  Engineering  V-model 


(generalized) 
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r  “Integrated”  System  Safety  Model 

_ (from  Defense  Acquisition  University  Course  CLE009) 


Classic  System  Safety  Model 


1 


Risk  Assessments 
FE3HE  at  each  Phase/Milestone 


* 

ID  Mishap 
Mitigation 
Measures 

* 

Reduce  Risk  to 
Acceptable 
Level 

Verify  Mishap 
Reduction 


Residual  Risk 
Acceptance 

t 

Track  Hazards, 
Residual  Risk 
&  Monitor 


System  Safety  Plan 
HRI  Matrix 

Risk  Acceptance  Authority 


PUL  PHA 
SSHA  SNA 
O&SHA 


Assessment  of 
Mishap  Risk 


Traceability  of 
Safety 

Requirements 


As  of:  2¥  Sep  (14 


System  Safety  Requirements 
(e.g.  GPV¥S)  Desjgn//Technology 
Alternatives 


Phase  Specific  Safety 
Design  Requirements 


Phase  Specific  Safety 
Test  Requirements 
component,  subsystem*  system 


"1 ►  Accepted/documnented  Risks 

(updated  risk  assessments)  Specific  to  Phase 


Hazard  Reporting 
Surveillance  Program 
Mishap  Reporting 


ESOH  Requirements 
Review  Baseline  Requirements 
Lessons  Learned 
SRGA 

Alternate  Designs 

Adv.  Technology  Risk  & 

Support  Approach 


System  Safety 
Approach 


K>  Hazards 
(Causal  Factors) 
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Safety 

Requirements 


SIR/ICE 
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v  “Integrated”  System  Safety  Model 


Classic  System  Safety  Model 

Inputs  Outputs  - ►  inputs 


1 


Risk  Assessments 
PE3HE  at  each  Phase/Milestone 


Traceability  of 
Safety 

Requirements 


* 

ID  MJshap 
Mitigation 
Measures 

t 

Reduce  Risk  to 
Acceptable 
Level 

Verify  Mishap 
Reduction 


Residual  Risk 
Acceptance 

t 

Track  Hazards, 
Residual  Risk 
&  Monitor 


System  Safety  Plan 
HRl  Matrix 

Risk  Acceptance  Authority 


PUL  PHA 
SSHA  SNA 
O&SHA 


Assessment  of 
Mishap  Risk 


System  Safety  Requirements 
(e.g.  GPV¥S)  Desjgn//Technology 
Alternatives 


Phase  Specific  Safety 
Design  Requirements 


Phase  Specific  Safety 
Test  Requirements 
component,  subsystem*  system 


"1 ►  Accepted/documented  Risks 

(updated  risk  assessments)  Specific  to  Phase 


Hazard  Reporting 
Surveillance  Program 
Mishap  Reporting 


Compare 


ESOH  Requirements 

Re  view  Baseline  Requirements 

Lessons  Learned 

SRGA 

Alternate  ensigns 

Adv.  Technology  Risk  ^ 

Support  Approach 


System  Safety 
Approach 


K>  Hazards 
(Causal  Factors) 


Assess 


Hazards 


Develop 

Mitigation 


Implement 

Mitigation 


Verify  Risk 
Reduction 


Yes 


Accept 

Risk 


Monitor 

Hazards 
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Safety 

Requirements 


SIR/ICE 
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r  “Integrated”  System  Safety  Model 


Classic  System  Safety  Model 


Areas 


Interest 


As  of:  2¥  Sep  (14 


D  Hazards 
(Causal  Factors) 


Reduce  Risk  U 
Acceptable 
Level 


Risk  Assessments 
RES  HE  at  each  Phase,'1  Mile  stone 


~r  Accepted/documented  Risks 
(updated  risk  assessments)  Specific  to  Phase 


Verify  Mishap 
Reduction 


Residual  Risk 
Acceptance 

t 

Track  Hazards, 
Residua]  Risk 
&  Monitor 


ESGH  Requirements 
Review  Baseline  Requirements 
Lessens  Learned 
SRGA 


Alternate  Designs 
My,.  Technology  Risk 
Support  Approach 


Assessment  of 
Mishap  Risk 


▼ 

ID  Mishap 
Mitigation 
Measures 


System  Safety  Rl!an 
HRl  Matrix 

Risk  Acceptance  Authority 


FHL  PHA 
SSHA  SNA 
O&SHA 


System  Safety  Requirements 
(e  gP  GPV¥S)  Desjgn/fFechnology 
Alternatives 


Hazard  Reporting 
Surveillance  Program 
Mishap  Reporting 


Traceability  of 
Safety 

Requirements 


Phase  Specific  Safety 
Design  Requirements 


Phase  Specific  Safety 
Test  Requirements 
component,  subsystem,  system 


System  Safety 
Approach 
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Safety 

Requirements 


SIR/ICE 
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System  Safety  Requirements 


*  Phase  Specific 

*  Managed  with  Other  System  Engineering  Artifacts 

-  Requirements  Traceability  (requirements  tool) 

-  CONOPS,  Conceptual  Design  &  System  Architecture 

-  Verification  and  Validation  Tests  (e.g.,  TEMP) 

*  Part  of  Technical  Baseline  for  Each  Phase 

-  Alternative  System  Review 

-  System  Functional  Review 

-  System  Requirements  Review 

-  Preliminary  Design  Review 

-  Critical  Design  Review 

-  Test  Readiness  Review 


NDIA  8th  Annual  Systems  Engineering  Conference 


17 
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System  Safety  Requirements 


*  Phase  Specific 

*  Managed  with  Other  System  Engineering  Artifacts 

-  Requirements  Traceability  Matrix 

-  CONOPS,  Conceptual  Design  &  System  Architecture 

-  Verification  and  Validation  Tests  (e.g.,  TEMP) 

*  Part  of  Technical  Baseline  for  Each  Phase 


-  Alternative  System  Review 

-  System  Functional  Review 

-  System  Requirements  Review 

-  Preliminary  Design  Review 

-  Critical  Des  Review 

-  Test  Reacy  xeview 


Figure  i  -  System  Engineering  V-niodelJ 


Somewhere  just  before 
here  is  typical  entry  point!! 
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Integrated  Systems  Engineering 

“The  Wall  Chart” 


Integrated  Defense  Acquisition,  Technology,  &  Logistics  Life  Cycle  Management  Framework 


Joint 

Capabilities 
In  Leg  ration  & 
Development 
System 


Defense 

Acquisition 

System 


Teebnie#! 


Let’s  focus  here 
for  a  moment 


Planningt 

Programming. 

Budgeting, 

&  Execution 
Process 


ly-jitm 
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Life  Cycle  Framework  In-service 
System  Safety  Requirements 


fere 
-  SEP 
er  Esf. 


INPUTS 


OUTPUTS 


itB 

1 


■  Service  Use  Date 
•  User  Fee  dback 
-Failure  Reports 
•Discrepancy  Reports 
■SEP 


Monitor  and  Coiled 
All  Servioe 
Use  Data 


-  Data  for  in -Service  Review 
•input  to  CDD  for  next  increment 
•Moditfceiionsiupgredes  to  fielded 
systems 
■ SEP 


■  Trades 


Implement  -and 
Field 


n 


Analyze  Data  to 
Determine 

Root  Cause 

o 

Determine 

System  R  sk,' 
Hazard  Severity 

Assess  Risk  of 
Improved  System 


r> 


Integrate  &  Test 
Corrective  Action 


Develop 

Corrective 

Action 


Process  Clhamge  -  Hardware^  uppo-rt 
Materiel  Change 


Important! 
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ENGINEERING  COMPANY 


Conclusions 


*  Requirements,  Requirements,  Requirements 

-  The  language  of  the  systems  &  design  engineers 

*  Integration  of  System  Safety  into  System 
Engineering  Framework  is  Critical 


•  Framework  is  the  Key 

*  Conditions  are  Right  (OSD  is  an  Advocate) 


•  Must  Understand  and  Spread  the  Word _ 

F«ctdine  Pliaw  Subsequent  Ruse 

V  £ 

V  ilii'  ili  in  ''  'l 

To  be  an  Effective  System  Safety  Practitioner, 
You  Must  Absolutely  Understand  and  Speak 
the  Systems  Engineering  Process!! 

rig,ni'*  3  -  System  Eng  meeting  V -mode  I4 


Integrated  Defense  Acquisition.  Technology.  &  Logistics  Life  Cycle  Management  Framework 
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The  Return  of  Discipline 


From  Operational  Safety,  Suitability  and  Effectiveness  (OSS&E) 

To 

Systems  Engineering  Implementation  at  Air  Force  Materiel  Command 


Presented  By:  Jackie  Townsend 

HQ  AFMC/ENP 
Wright-Patterson  AFB,  OH 
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Overview 


■  OSB&E  Policy  History 

■  Policy  Execution 

■  OSS&E  Implementation  Efforts 

■  Three  "R"'s  of  Systems  Engineering 

■  Way  Forward 


OSS&E  Policy  History 


■  Series  of  aircraft  mishaps/ 
incidents 

■  Loss  of  Discipline 

-  Loss  of  Configuration  Control 

-  Incomplete  or  outdated  Technical  Data 

-  Unqualified  People  or  Organizations  making  modifications/changes 

-  Unauthorized  changes 

-  Lack  of  or  incomplete  testing 

-  Improper  procedures/procedures  not  followed 

-  Lack  of  interface  controls 

-  Improper  integration 


OSS&E  Policy  History 

■  AFMC  Response 

-  Discussed  with  CSAF  /  SecAF  and  Subsequent  Direction 

-  Established  Policies  for  Preserving  Operational  Safety, 
Suitability  &  Effectiveness  (OSS&E) 

>  Published  AFPD  63-12,  AFI  63-1201  &  AFMCI  63-1201 

-  Preserve  Established  Baseline  Characteristics  Throughout 
Operational  Life  of  a  System  or  End-Item 

-  Designate  Responsibility  and  Authority 

-  Use  Disciplined  Processes 

-  Maintain  Baselines  Throughout  Operational  Life 


Essentially... Systems  Engineering  + 
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■  OSS&E  Implementation  Efforts 
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■  Way  Forward 


Policy  Execution 


Assure  OSS&E ...  employ  disciplined  processes 
and  effective  procedures 


OBJECTIVES 

•  Deliver  systems/ 
end-items  with 
OSS&E  baseline 

•  Preserve  the 
baseline  over 
system  life 

•  Update  baseline 
when  making 
modifications  or 
changes 


REQUIRED  PROCESSES/PROCEDURES 

•  Disciplined  systems  management 

•  Disciplined  systems  engineering 

•  ORM 

•  Systems  safety 

•  Config  mgmt 

•  Certifications 

•  Effective  ops  procedures 

•  Effective  training 

•  Effective  supply,  inspection,  and 
maintenance  procedures 

•  Quality  sources  of  supply, 
maintenance,  and  repair 


Policy  Execution 


The  policies  require  the  preservation  of 
operational  safety,  suitability,  and 
effectiveness  baseline  characteristics  of 
delivered  systems  and  end-items  over 
their  operational  life 


Policy  Execution 


The  policies  require  the  preservation  of 
operational  safety,  suitability,  and 
effectiveness  baseline  characteristics  of 

systems  and  end-items  over 
their  operational  life 


Became  an  "engineering"  focus... and  at 
times.. .an  engineering  sustainment  focus 


Single  Manager/Chief  Engineer  must  know 
the  answers  to  the  tough  questions... 


What  parts  are 
going  obsolete'L 


What  are  the  certification 
requirements? 


How  is  the  threat  changing? 


What  are  the  Operational 
Requirements? 


Who  is  making  replacement 
decisions  in  my  supply  chain?. 


What  is  the  fielded 
configuration  of  my 
system/end-item? 


Who  buys  my  Spares  and 
what  change  authority  has 
been  delegated  to  them'L 


What  T.O.s  are  fielded 
and  are  they  current? 


Who  does  my  maintenance? 
What  change  authority  has 
been  delegated  to  them? 


How  is  my  system  aging? 
How  is  it  performing? 


What  modifications  are  being  made? 
How  do  they  impact  my  systems? 


How  many  of  what  type 
is  fielded  (  Serial  #  &  Tail  t) 
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HQ  AFMC  OSS&E 
Implementation  Levels 


Level  1  -  Chief  Engineer  Assigned 

Level  2  -  Configuration  Control  Processes  Established 

Level  3  -  Plan  to  Assure  and  Preserve  OSS&E  Documented 

Level  4  -  OSS&E  Baselines  Developed  and  Coordinated  with  User 

Level  5  -  OSS&E  Assessment  of  Fielded  Systems  and/or  End  Items 

Level  6  -  Full  OSS&E  Policy  Compliance 


Driving  the  wrong  behavior... change  is  needed 
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Three  "R"'s  of  Systems 
Engineering 


■  R  Processes/Policies 

■  R  Technical  Rigor 

■  R  Strategy 


R  Processes/Policies 

Systems  Engineering  Policy-Vision 


DoDD  5000.1 


DoDI  5000.2 


DoD 


AF 


Defense 

Acquisition 

Guidebook 


USD/AT&L  Policy 
Letter  Policy  for  SE 
in  DoD 


NSS- 

03-01 


1 

"*AFPD  63-1 

Quality 

Maintenance 

Capability-Based 
ition : 

Assurance 

Systems  Eng 

Software 

Engineering 

Human  Sys 
Integration 


AFMC 


AFI  63-101 

Acquisition  System 


Manufacturing 


AFI  63-107 

Product  Support 
Planning/Ass 


AFMCI  63-xx 


AFI  63-1  xx 

Systems  Engineerii 


Maintain 

OSS&E 


AEDC  I  AFFTC 


OC-ALC  ■  OO-ALC  ■  WR-ALC 


Programs 


Program  SEP 

Program  SEP 

Program  SEP 

&SEMP 

&SEMP 

&SEMP 

R  Technical  Rigor 

4  Development  of  technical  integrity 
handbook/training 

«  Requirement  for  Systems  Engineering 
Plan 

4  Publishing  of  key  criteria  for  engineers 

■  Establishment  of  clear 
standards/metrics 

■  Alignment  of  SE  and  OSS&E 


Strategy 


■  Establishment  of  clear 
standards/metrics 

■  Alignment  of  SE  and  OSS&E 

•SE  AFI  (The  Role  of  Systems  Engineering) 
•OSS&E  (SE  Process  Assurance  Standard) 
•Standardized  Reporting 
•Training 


Overview 


■  OSS&E  Policy  History 

■  Policy  Execution 

■  OSS&E  Implementation  Efforts 

■  Three  "R"'s  of  Systems  Engineering 

■  Ray  Forward 


Integrated 


Top-level 

policy 


SE  AFI  -  The  Role 
^  Of  System  Engrg 

Core  Elements 

•  Requirements 

•  Planning 
•CM 


•  Risk/Safety 

•  Interop 

•  Sys  Mgmt 

•  Ops  Procs 

•  Quality  Sources 

•  Software 


APP 

•  Other  elements 


Approach 


OSS&E 

Assurance  Standards 

Standards 

•  Requirements 

•  Planning 
•CM 

•  Risk/Safety 

•  Interop 

•  Sys  Mgmt 

•  Ops  Procs 

•  Quality  Sources 

•  Software 

•  RTOC 

•  Standards  for 
OSS&E  baselines 
&  reporting 

APP 

•  Stds  for  other 
elements 


Standards 
for  OSS&E 


Update  core 
elements  based 
on  SE  AFI,  Key 
elements... 


Integrated  Approach 


SE  AFI  -  The  Role 
Of  System  Engrg 

Core  Elements 

•  Requirements 

•  Planning 
•CM 

•  Risk/Safety 

•  Interop 

•  Sys  Mgmt 

•  Ops  Procs 

•  Quality  Sources 

•  Software 


OSS&E 

Assurance  Standards 

Standards 

•  Requirements 

•  Planning 
•CM 

•  Risk/Safety 

•  Interop 

•  Sys  Mgmt 

•  Ops  Procs 

•  Quality  Sources 

•  Software 

•  dt nr 

IX  I  v/v 

•  Standards  for 
reporting  OSS&E 


Training  -  Systems 
Engineering  &  OSS&E 

Core  Elements 

•  Requirements 

•  Planning 
•CM 

•  Risk/Safety 

•  Interop 

•  Sys  Mgmt 

•  Ops  Procs 

•  Quality  Sources 

•  Software 

•  Standards  for 
reporting  OSS&E 


Where  we  re  headed 


1998 


1999 


2000 


2001 


2002 


2003 


2004 


CSAF  Memo 
Tasked  Interim 


AFPD  62-6 
A/C  Airworthiness 
/  OctOO^ 


AFPD  63-13 
GATM  and 
Nav  Safety 
Certif  /  MarOI 


AFPD  62-5 
Airworthiness 
Derivative  A/C 
AugOI 


Certifications 


AFI  63-125 
Nuclear  Certif 
Program  /  Mar04 


km  yn  siiiii 


99 

ASC  Cl 
To  Dev 


The  Return  of  Discipline 


Product 

Centers 


SMC  &  ESC 

AACI  63-1201  CM  EngrS 
(draft)  SMCI 63-1201 

ESCI  63-1201 

♦  ♦  ♦ 


ASC  handbook 


ESC  Handbook  / 

MIL  HDBK-514 


FI  21-101 


IXtomm 


AFI  91-204 
AFI  91-222 


OO-ALC 


ALCs 


OC-ALC/CC 

Memo  OPR  and 
Procedures 


Established 


OC-ALC 

Tl  63-101 


Safety 


Operational  metrics: 
Air  Worthiness 
Certification, 

Mishap  Risk,  Loss  Rate... 


With  the  operational  life 
of  weapon  systems  often 
extending  50-70  years, 
preservation  of  combat 
capability  is  essential. 
The  policy  aims  to  ensure 
the  same  low  level  of 
safety  risk  we  boast  at 
fielding  is  maintained 
across  the  operational  life. 


Suitability 


Operational  metrics: 

Mission  Capability  Rate, 
MTBF,  MTBF,  MTTR,  ... 


The  policy 
ensures  that  as 
the  "systems  of 
systems" 
architecture 
changes,  the 
weapon  system 
will  remain 
equally  suited  to 
the  task 


Effectiveness 


Operational  metrics: 

Range,  Payload,  Cargo 
Capability, ... 


It  also  ensures  the 
effectiveness  of  the 
system  as  far  as 
accuracy,  endurance, 
etc.,  remains  constant 
over  the  years.  To 
accomplish  these  ends, 
clear  responsibilities 
are  established  both  for 
the  single  manager, 
AFMC  and  the  using 
commands. 


Decision  Analysis  and 

Resolution 

Tailorable  Decision  Analysis  & 
Resolution  process  and  tools  for 
enterprise  wide  application 


Enabling  the  American  Warfighter  to  Dominate  the  Battlefield! 


Outline 

•  Introduction 

•  ARDEC  Systems  Engineering 

•  Decision  Analysis  and  Resolution  (DAR) 

•  DAR  Process 

•  Tailored  application  of  DAR  to  Technical 
Trade  Study 

•  Benefits 


Enabling  the  American  Warfighter  to  Dominate  the  Battlefield! 


ARDEC  Background 


Artillery  & 
Mortar 
Systems 

DEMIL 


R&D 


Advanced  Fuze 
Technologies 


Smart  Muniti 


SUPPOR 
TOTAL 
LIFE 
CYCL 


PROD 


FIELD  SUPPORT 


Special  Operations 
Weapons  &  Demolitions 


Advanced  Explosives  & 
Warhead  Development 


Combat  Vehicle 
Armaments  &  Fire  Control 


Logistics  R&D 


Non-Lethal 

Technologies 


Future  Small  Arms 


PROVIDING  OVER  90%  OF  THE  ARMY’S  LETHALITY. .. 


Introduction 


•  ARDEC  Systems  Engineering  (SE)  Division 

-  Established  from  ARDEC  re-organization  to  focus  on  disciplined 
systems  engineering 


“System  Engineering  objectives  provides  the 
integrating  technical  process  to  define  and 
balance  system  performance,  cost,  schedule, 
and  risk.” 

-Michael  W.  Wynne 

Acting  Under  Secretary  Of  Defense 

20  Feb.  2004 


•  System  Engineering  (SE)  Process  needed  a  consistent 
and  effective  process  for  making  fact  based  decisions. 


Enabling  the  American  Warfighter  to  Dominate  the  Battlefield! 


Decision  Analysis  and  Resolution 

Definition 

•  Analyze  possible  decisions  using  a  formal 
evaluation  process  that  evaluates 
identified  alternatives  against  established 
criteria. 


Enabling  the  American  Warfighter  to  Dominate  the  Battlefield! 


Decision  Analysis  and  Resolution 

Impact 

•  Inconsistent  DAR  processes  may... 

-  Cause  delays/bottlenecks  when  reviewers 
inquire  how  the  decision  came  to  being. 

-  Raise  the  learning  curve  of  new  IPTs  (must 
agree  on  common  ones). 

-  Not  reach  the  best  achievable  solution. 


Enabling  the  American  Warfighter  to  Dominate  the  Battlefield! 


Approach 

•  Used  as  Six-Sigma  Green  Belt  Project  -  Major 
Initiative  at  ARDEC  to  use  Lean/6-Sigma 

•  Methodologies/Tools  Used 

-  Brainstorming 

-  Process  Map 

-  Voice  of  the  Customer 

-  FMEA 

-  Quality  Function  Deployment 

-  Product  Selection  Matrix 


Enabling  the  American  Warfighter  to  Dominate  the  Battlefield! 


Level  1  Decision  Analysis  and  Resolution  Process 


Define  Problem 
100 


Decide  if  Formal 
Evaluation 
Process  is 
Necessary 
200 


Define  Customer 

Customer  Problem  Statement 

Requirements 

Operational  Needs 

Approval  Authority 

Brainstorming 


Define  Project 
Plan 
300 


Select  Evaluation 
Methods 
400 


Establish  Required 
Evaluation  Criteria 
(For  Filter) 

500 


Multi-Attribute  Utility 
Theory 
Expert  Matrix 
QFD 

Reporting  Method 


Collect  Customer 

Requirements 

Define  Required  Criteria 

for  First  Cut 

Define  Evaluation 

Metrics 

Capture  Data 


Establish  Desired 

Establish  Desired 
Criteria 

Evaluation  Criteria 
600 

Importance 

700 

Collect  Customer 
Requirements 

Define  Desired  Criteria 
for 

Define  Evaluation 

Metrics 

• 

Define  Importance  of 
Criteria 

Create  Weighting 
Structure  Data 

AHP 

Desirability  Curves 

Capture  Data 

Enabling  the  American  Warfighter  to  Dominate  the  Battlefield! 


Level  1  Decision  Analysis  and  Resolution  Process 


Define  Problem 
100 


Decide  if  Formal 
Evaluation 
Process  is 
Necessary 
200 


Define  Customer 

Customer  Problem  Statement 

Requirements 

Operational  Needs 

Approval  Authority 

Brainstorming 


Define  Project 
Plan 
300 


Select  Evaluation 
Methods 
400 


Establish  Required 
Evaluation  Criteria 
(For  Filter) 

500 


Multi-Attribute  Utility 
Theory 
Expert  Matrix 
QFD 

Reporting  Method 


Collect  Customer 

Requirements 

Define  Required  Criteria 

for  First  Cut 

Define  Evaluation 

Metrics 

Capture  Data 


Establish  Desired 
Evaluation  Criteria  - 
600 


Collect  Customer 
Requirements 
Define  Desired  Criteria 
for 

Define  Evaluation 
Metrics 
Capture  Data 


Establish  Desired 
Criteria 
Importance 
700 

Define  Importance  of 
Criteria 

Create  Weighting 
Structure  Data 
AHP 

Desirability  Curves 


Evaluation 

Criteria 


Define  Alternatives 
800 


Collect  Alternatives 
Determine 

Applicability  Set  of 


Research 

Candidate 

Alternatives 

900 

Score  Alternatives 
against  Required 
Criteria 

1000 

/Z, Alternatives^\ 

V  Still  Exist 

.  •  Collect  Data 

1 

r 

•  First  Cut  Filter 

Test 

M&S 


Data  for 
Alternatives 


Score  Alternatives 
against  Desired 
Criteria 
1100 


Analyze  and  Test 
Results 
1200 


Refined  Set  of 
Alternatives 


Using  defined 
scoring  method 


Calculated 

Alternatives’ 

Score(s) 


Perform  Sensitivity 
Analysis 

Monte  Carlo  Analysis 


Select  /  Rank 
Alternatives  for 
Approval 
1300 


Validated  Set  of 
Results 


▼ 

Best 

Alternative(s)  for 
Approval 
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Enabling  the  American  Warfighter  to  Dominate  the  Battlefield! 


Level  1  Decision  Analysis  and  Resolution  Process 


Operational  Needs 
Approval  Authority 
Brainstorming 


Reporting  Method 


Establish  Desired 
Evaluation  Criteria  - 
600 


Establish  Desired 
Criteria 
Importance 
700 


Establish  Required 
Evaluation  Criteria 
>  (For  Filter) 

500 


Collect  Customer 

Requirements 

Define  Required  Criteria 

for  First  Cut 

Define  Evaluation 

Metrics 

Capture  Data 


Collect  Customer 
Requirements 
Define  Desired  Criteria 
for 

Define  Evaluation 
Metrics 
Capture  Data 


Define  Importance  of 
Criteria 

Create  Weighting 
Structure  Data 
AHP 

Desirability  Curves 


Filter  Criteria 


Evaluation 

Criteria 


I 


No 


No 
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Yes- 


Define  Project 
Plan 
300 


Schedule 


> 


Select  Evaluation 
Methods 
400 


Multi-Attribute  Utility 
Theory 
Expert  Matrix 
QFD 

Reporting  Method 


Establish  Required 
Evaluation  Criteria 
(For  Filter) 

500 


Collect  Customer 

Requirements 

Define  Required  Criteria 

for  First  Cut 

Define  Evaluation 

Metrics 

Capture  Data 


Filter  Criteria 


Establish  Desired 
Evaluation  Criteria 
600 


Collect  Customer 
Requirements 
Define  Desired  Criteria 
for 

Define  Evaluation 
Metrics 
Capture  Data 

T 

Evaluation 

Criteria 


Establish  Desired 
Criteria 
Importance 
700 


Define  Importance  of 
Criteria 

Create  Weighting 
Structure  Data 
AHP 

Desirability  Curves 
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Level  1  Decision  Analysis  and  Resolution  Process 


Define  Project 
Plan 
300 


Select  Evaluation 
Methods 
400 


Multi-Attribute  Utility 
Theory 
Expert  Matrix 
QFD 

Reporting  Method 


Establish  Required 
Evaluation  Criteria 
(For  Filter) 

500 


Collect  Customer 

Requirements 

Define  Required  Criteria 

for  First  Cut 

Define  Evaluation 

Metrics 

Capture  Data 


Establish  Desired 
Evaluation  Criteria  - 
600 


Collect  Customer 
Requirements 
Define  Desired  Criteria 
for 

Define  Evaluation 
Metrics 
Capture  Data 


Establish  Desired 

Criteria  _ 

Importance 

700 

Define  Importance  of 
Criteria 

Create  Weighting 
Structure  Data 
AHP 

Desirability  Curves 


Evaluation 

Criteria 


1 


No 
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Define  Alternatives 
800 

Research 

Candidate 

W 

Alternatives 

900 

• 

• 

Collect  Alternatives  i 
Determine 

r 

•  Collect  Data 

•  Test 

Applicability 

Set  of 
Alternatives 

•  M&S 

Data  for 
Alternate 
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Operational  Needs 
Approval  Authority 
Brainstorming 


Reporting  Method 


Establish  Desired 
Evaluation  Criteria  - 
600 


Establish  Desired 
Criteria 
Importance 
700 


Establish  Required 
Evaluation  Criteria 
>  (For  Filter) 

500 


Collect  Customer 

Requirements 

Define  Required  Criteria 

for  First  Cut 

Define  Evaluation 

Metrics 

Capture  Data 


Collect  Customer 
Requirements 
Define  Desired  Criteria 
for 

Define  Evaluation 
Metrics 
Capture  Data 


Define  Importance  of 
Criteria 

Create  Weighting 
Structure  Data 
AHP 

Desirability  Curves 
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Enabling  the  American  Warfighter  to  Dominate  the  Battlefield! 


Level  1  Decision  Analysis  and  Resolution  Process 


Define  Project 
Plan 
300 


Select  Evaluation 
Methods 
400 


Multi-Attribute  Utility 
Theory 
Expert  Matrix 
QFD 

Reporting  Method 


Establish  Required 
Evaluation  Criteria 
(For  Filter) 

500 


Collect  Customer 

Requirements 

Define  Required  Criteria 

for  First  Cut 

Define  Evaluation 

Metrics 

Capture  Data 


Establish  Desired 
Evaluation  Criteria  - 
600 


Collect  Customer 
Requirements 
Define  Desired  Criteria 
for 

Define  Evaluation 
Metrics 
Capture  Data 


Establish  Desired 

Criteria  _ 

Importance 

700 

Define  Importance  of 
Criteria 

Create  Weighting 
Structure  Data 
AHP 

Desirability  Curves 


Evaluation 

Criteria 


No 


r 
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Enabling  the  American  Warfighter  to  Dominate  the  Battlefield! 


Operational  Needs 
Approval  Authority 
Brainstorming 


Reporting  Method 


Establish  Desired 
Evaluation  Criteria  - 
600 


Establish  Desired 
Criteria 
Importance 
700 


Establish  Required 
Evaluation  Criteria 
>  (For  Filter) 

500 


Collect  Customer 

Requirements 

Define  Required  Criteria 

for  First  Cut 

Define  Evaluation 

Metrics 

Capture  Data 


Collect  Customer 
Requirements 
Define  Desired  Criteria 
for 

Define  Evaluation 
Metrics 
Capture  Data 


Define  Importance  of 
Criteria 

Create  Weighting 
Structure  Data 
AHP 

Desirability  Curves 
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Enabling  the  American  Warfighter  to  Dominate  the  Battlefield! 


Level  1  Decision  Analysis  and  Resolution  Process 


Define  Project 
Plan 
300 


Select  Evaluation 
Methods 
400 


Multi-Attribute  Utility 
Theory 
Expert  Matrix 
QFD 

Reporting  Method 


Establish  Required 
Evaluation  Criteria 
(For  Filter) 

500 


Collect  Customer 

Requirements 

Define  Required  Criteria 

for  First  Cut 

Define  Evaluation 

Metrics 

Capture  Data 


Establish  Desired 
Evaluation  Criteria  - 
600 


Collect  Customer 
Requirements 
Define  Desired  Criteria 
for 

Define  Evaluation 
Metrics 
Capture  Data 


Establish  Desired 

Criteria  _ 

Importance 

700 

Define  Importance  of 
Criteria 

Create  Weighting 
Structure  Data 
AHP 

Desirability  Curves 


Evaluation 

Criteria 


Define  Alternatives 
800 


Research 

Candidate 

Alternatives 

900 


Collect  Alternatives  ▼ 
Determine 

Applicability  Set  of 


Collect  Data  f  • 
Test  • 

M&S  Data  for 

Alternatives 


Score  Alternatives 
against  Required 
Criteria 
1000 

•  First  Cut  Filter 

•  Yes/No 


Score  Alternatives 
against  Desired 
Criteria 
1100 


Refined  Set  of 
Alternatives 


Using  defined 
scoring  method 


Calculated 

Alternatives’ 

Score(s) 


Analyze  and  Test 
Results 
1200 


Perform  Sensitivity 
Analysis 

Monte  Carlo  Analysis 


No 


r 
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Select  /  Rank 
Alternatives  for 
Approval 
1300 


Best 

Alternative(s)  for 
Approval 


Identify  deficiency  in  criteria/alternatives 
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Operational  Needs 
Approval  Authority 
Brainstorming 


Reporting  Method 


Establish  Desired 
Evaluation  Criteria  - 
600 


Establish  Desired 
Criteria 
Importance 
700 


Establish  Required 
Evaluation  Criteria 
>  (For  Filter) 

500 


Collect  Customer 

Requirements 

Define  Required  Criteria 

for  First  Cut 

Define  Evaluation 

Metrics 

Capture  Data 


Collect  Customer 
Requirements 
Define  Desired  Criteria 
for 

Define  Evaluation 
Metrics 
Capture  Data 


Define  Importance  of 
Criteria 

Create  Weighting 
Structure  Data 
AHP 

Desirability  Curves 
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ARDEC  Enterprise 
Application 

•  DAR  process  to  be  approved  as  part  of 
formal  ARDEC  SE  Standard  Process 

•  Projects  are  required  to  tailor  and  use 
process  for  their  application 

•  Identified  methodologies/tools  for  each 
process  step  to  facilitate  process 
execution 
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Tailored  DAR  Process 


•  Application  for  FCS  Active  Protection 
System  (APS)  Technical  Trade  Study  for 
RDECOM. 

-  Identify  Science  and  Technology  Investments 
needed  to  get  to  an  objective  APS  system. 

•  ARDEC  DAR  process  focused  competing 
organizations’  efforts  to  determine  path- 
forward  for  the  APS  technical  trade  study 
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Tailored  DAR  Process 


Start 


Develop  System 

Develop  System 

Stakeholder 
Concurrence  on 

Level  Evaluation 
Criteria 

Level  Criteria  Wts 
&  Filter 

System  Evaluation 
Criteria,  Wts  & 
Filter 

Develop  DAR 
Process/Approach 


D  -  change  from 
original  process 


Develop 

Develop 

Stakeholder 

Component  Level 
Evaluation  Criteria 

Component  Level 
Coarse  Filter 

Concurrence  on 
Component 
Coarse  Filter 

Research  System 
Candidates 


Develop  Alternate 
System  Sol’ns 
based  on 
Component 
Alternatives 


Develop 

Component 

Compatibility 

Matrix 


Conduct 

Component  Level 
Sensitivity 
Analysis 


Develop 

Component  criteria 
&  Score 

Remaining  ◄- 
Component 
Alternatives 
Against  Criteria 


Yes 


May  not  be  required  depending 
on  #  of  viable  technology  sol’ns 


Score  Remaining 
System 

Conduct  System 
Level  Sensitivity 
analysis 

ID  &  Rank  Viable 

Alternatives 
Against  Criteria 

w 

w 

System  Sol’ns 

W* 

Yes 


End 
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DAR  Benefits 


•  Project  risk  will  be  reduced  by  applying  the 
defined  DAR  process. 

•  Fact-based  decision  will  be  made  rather 
than  subjective  decisions. 

•  Increased  quality  decisions 

-  Defendable 

-  Stakeholder  buy-in 

-  Flexible 
-Valid 
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Return  on  Investment 


Quality  or  Customer  Satisfaction 

o 

o 

•DAR  Process  created  to  enhance  the 
ARDEC  capability  to  deliver  quality 
products  to  the  Warfighter. 

•Tools  capability  to  support  ARDEC  project 

execution 

Savings  from. . . 

•Reuse 

•Standardization 

•Best  Practice  application 
•  Savings:  $11.3K/use 

•Defined  DAR  process  provides  for  better 
time  and  resources  scheduling  needed  to 

execute. 

•Lowers  the  learning  curve  DAR  application 

•Defined  DAR  process  reduces  risk  by 
providing  a  tailorable  framework  for  making 

decisions 

Schedule 

Risk 
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•  Questions? 
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Effective  SE  Metrics 
Tailored  to  the 
Acquisition  Life  Cycle 


Armament  Research,  Development  &  Engineering  Center 

Armament  System  Integration  Center 
Systems  Engineering  Division 


Laura  Troiola 

Systems  Engineering  Advisor 

ltroiola@pica.armv.mil 

(973)  724-6296 
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AGENDA 


K*f 
Zt 
Ui 

^gdnteg  ratmg  1  ecnjaoiogv^y 
With  The  Soldier 

"V  a3^ 


•  ARDEC  Background 

•  Measurement  Approaches 

-  Systems  Engineering  Plan 

-  Level  of  Effort  Assessment 

•  Tracking  &  Reporting 

•  Benefits 

•  Next  Steps 
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ARDEC  Background 


Advanced  Fuze 
Technologies 


Artillery  & 
Mortar 
Systems 


R&D 


DEMIL 


Smart  Muniti 


SUPPOR 
TOTAL 
LIFE 
CYCL 


PROD 


FIELD  SUPPORT 


Special  Operations 
Weapons  &  Demolitions 


Advanced  Explosives  & 
Warhead  Development 


Combat  Vehicle 
Armaments  &  Fire  Control 


Logistics  R&D 


Non-Lethal 

Technologies 


Future  Small  Arms 


PROVIDING  OVER  90%  OF  THE  ARMY’S  LETHALITY ... 


Planned  versus  Actual 

Metric:  SE  Planning 

•  Purpose 

-  Living  Document  for  Planning 

-  Drive  Technical  Execution 

•  Rolling  Wave  Concept 

•  Tailoring 

-  Based  on  Acquisition  Phase 

-  Project  Specific  Technical  Activities 

•  Level  of  Risk  Acceptance 

-  Programmatic  Factors  to  Consider 

•  Resources 

•  Complexity 

•  Customer  &  Stakeholders  Needs 

•  Schedule 


^integrating  t  eeiyioiog^y 
With  The  Soldier 
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Metric:  Level  of  Effort  Assessment 


zl 

ui 

^gdnteg  ratmg  1  eengoiog^y 
With  The  Soldier 

a3^ 


•  Based  on  Acquisition  Phase 

•  Define  Project  SE  status  in  Key  Areas 

-  Requirements 

-  Functional  Analysis  &  Allocation 

-  Design  Synthesis 

-  Verification  &  Validation 

-  System  Analysis  &  Control 

•  Quantifies  Remaining  SE  Work  on  Project 

•  Traced  to  OSD  &  ARDEC  Guidance 

-  Defense  Acquisition  Guide 

-  Policies,  Process,  Procedures,  Templates 

•  Validated  with  Other  Factors  to  Consider 

•  Used  to  Develop  SE  Plans  and  Budgets 
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Other  Factors  to  Consider 


•  Funding 

•  Customer 

•  Stakeholders  &  End  User 

•  In-house  Work  Versus  Outsourced 

•  ARDEC  Priorities  and  Visibility 

•  Percent  Complete 

•  Resources  and  IPT  Members 

•  Technology  Complexity  &  Domain 

•  Other  Factors  the  Rater  Wants  SE  to  Consider 
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System  Development  and 
Demonstration  Phase 


JintegrafiftQ  1 
£-With  The  Soldier 


INPUTS 


OUTPUTS 


■r, 

•  F: 


Sys  Performance  Spec 
Exit  Criteria 

•  Validated  Sys  Support  & 
Maintenance  Objectives  & 
Requirements 
•APB  •  CDD  •  SEP 
ISP*  TEMP 


A 


Interpret  User  Needs, 
Refine  System 
Performance  Specs  & 
Environmental  Constraints 


Develop  System 
Functional  Specs  & 
System  Verification  Plan 


FCA 


•Initial  Prod  Baseline 
•Test  Reports  •  TEMP 
Elements  of  Product  Support 
•Risk  Assessment 
•SEP  -TRA  •  PESHE 
•Inputs  to: 

-CPD  - STA  -ISP 
-Cost/Manpower  Est.  ^ 

SVR 

Comfiined  DT&E/OT&E 
Demonstrate  System  to 
Specified  User  Needs  & 
Environmental  Constraints 


Trades u 


System  DT&E,  LFT&E  &  OAs, 
Verify  System  Functionality 
&  Constraints  Compliance 
to  Specs 


Evolve  Functional 
Performance  Specs  into 
Cl  Functional  (Design  to) 
Specs  and  Cl  Verification  Plan 


Evolve  Cl  Functional 
Specs  into  Product 
(Build  to)  Documentation 
and  Inspection  Plan 


TRR 


Integrated  DT&E,  LFT&E  & 
EOAs  Verify  Performance 
Compliance  to  Specs 


Individual  Cl 
Verification 
DT&E 


Fabricate,  Assemble, 
Code  to  “Build-to” 
Documentation 
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System  Development  &  Demonstration  :  Pre-Milestone  O 


SEL 

Project  Name 

Type  of  Program 
(A-F) 

1 

2 

3 

N/A  &  Rationale 

Key  Areas 

System  Engineering  Plan 

Drafted  Updated  Plan 

Submitted  Updated  Plan 

Approved  Updated  Plan 

I 

Interpret  User  Needs 

Do  not  have  defined 
requirements 

Develop  requirements  from 
lifecycle  considerations;  use 
prototypes  for  stakeholder 
buy-in 

Manage  system 
requirements;  address 
and  characterize  risk 
associated  with 
requirements;  conduct 
SRR  if  necessary 

Requirements  not  yet 
decomposed;  RM  started 

Utilized  RM  Tool 

Requirements  traced  in 
database/tool 

Refine  System  Performance  Specs 

Fundamental  understand 
performance  specs 

Documented  Performance 
Specs 

Refined  Performance 
Specs 

]|| 

Develop  System  Functional  Specs 
&  System  Verification  Plan 

Have  not  yet  developed 
subsystems 

Partition  the  system  into 
subsystems;  define 
subsystem  interfaces  and 
integration 

Developed  subsystem 
integration,  verification 
and  validation 
plan/process 

I 

f 

Evolve  Function  Performance 
Specs  into  Cl  Functional  Specs  & 

Cl  Verification  Plan 

Have  not  allocated  specs 
or  defined  Cl 
performance/ 
functional  requirements 

Allocate  system 
functional/performance 
specs;  functional/ 
performance  requirements 
defined  for  Cl 

Create  test  plan  for 
verification  of  Cl  for 
gffibctionality/ 
ltformance 

Evolve  Cl  Functional  Specs  into 
Product  Documentation  & 
Inspection  Plan 

Have  not  begun 
documentation  for 
"buildinq"  components  ^ 

Complete^M  A  ,  ■ 

krawings/i  #  'o  1  k  lilized  detailed 

'  ■  1  1  1  1  Bgn;  Completed  CDR 

k  ^  h  1  1 

Lklll' 

I 

6 

Success/Fail  Criteria 

Si  1  9  T 

nC  1  W  «  \  I  1 

^Aimented/Approved 
^Bess/fail  criteria 

Fabricate,  Assemble,  Code  to 
"Built-to"  Documentation 

-  JiM  N 

Si 

cc  1 
re 

1  fae:  l  Ls ; 

1  v  >  r  r  d  a  ..on: 

Hes  t tPR  r  n  at  i v  e 

My  if  needed 

Created 

prototypes/engineering 
development  models 

Individual  Cl  Verification  DT&E 

v<  \  .icfknon 

Assess  technical  progress 
against  critical  technical 
parameters 

Demonstrate 
characteristics  of 
components  to  be 
integrated 

Integrated  DT&E,  LFT&E,  EOAs 
Verify  Performance  Compliance  to 
Specs 

Have  not  planned  for 

TRR,  verification  & 
validation 

Conduct  test  and  evaluation 
at  subsystem  level;  Plan  for 
TRR 

Verified  subsystem 
performance  against 
defined  subsystem 
design  requirements; 
Validated  intended 
subsystem  use  in 
environment 

System  DT&E,  LFT&E,  Oas,  Verify 
System  Functionality  &  Constraints 
Compliance  to  Specs 

Have  not  worked  to 
resolve 

i  nte  rf  ace/i  nteg  ratio  n 
issues;  do  not  monitor 
integration  performance 
risks 

Resolve  interface  and 
integration  issues;  monitor 
and  analyze  risks  for 
performance  of  integrated 
system 

Demonstrate  integrated 
system  under 
operational  environment 
constraints 

Combined  DT&E/OT&E/LFT&E 
Demonstrate  System  to  Specified 
User  Needs  &  Environmental 
Constraints 

Do  not  understand 
interface  and 
interoperability  issues; 
have  not  defined  test 
environments/ 
scenarios 

Defined  developmental  and 
operational  test 
environments/scenarios 

Resolve 

interface/interoperability 
issues;  confirm 
operational  supportability 
and  manufacturing 
process  control;  assess 
technical  risk  and 
mitigate 

« 

DM  &  CM  Requirements 

Identify  DM  &  CM 
Requirements 

Develop  &  Maintain  DM  &  CM 
Requirements 

Maintain  DM  &  CM 
Requirements 

DM/CM  Tool(s)  that  meet  the 
DM/CM  Requirements 

Identify  DM/CM  Tool(s) 
that  meet  the  DM/CM 
Requirements 

Develop  DM/CM  Tool(s)  that 
meet  the  DM/CM 
Requirements 

Maintain  DM/CM  Tool(s) 
that  meet  the  DM/CM 
Requirements 

eate  Risk  Plan 

Identified  Risks 
(probabilities  & 
consequences/impact) 

Documented  Risk  Plan  with 
Mitigation  Strategy  & 
Corrective  Action  Plan 

Tracked  Risk  Plan  with 
Mitigation  Strategy  & 
Corrective  Action  Plan 

jin  teg  rating  1  eoncoiog  YjCA 
S^Wrth  The  Soldier 
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System  Development  &  Demonstration 

Requirements  Metrics 


Jin t eg  rating  !  eoncoiog  VjCJ 
S^With  The  Soldier 


Key  Areas 

1 

2 

3 

HI  A  &  Rationale 

Requirem  ents 

Interpret  User  Needs 

Do  not  have  defined 
requirements 

Develop  requirements  from 
lifecycle  considerations;  use 
prototypes  for  stakeholder 
buy-in 

Manage  system 
requirements;  address 
and  characterize  risk 
associated  with 
requirements;  conduct 
SRR  if  necessary 

Requirements  not  yet 
decomposed;  RM  started 

Utilized  RM  Tool 

Requirements  traced  in 
database/tool 

Refine  System  Performance  Specs 

Fundamental  understand 
performance  specs 

Documented  Performance 
Specs 

Refined  Performance 
Specs 
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Requirements 


System  Development  &  Demonstration 

Requirements  Metrics 

EXAMPLE 


Jin t eg  rating  !  eoncoiog  VjCJ 
S^With  The  Soldier 

a# 


Key 

Areas 


1 

2 

3 

N/A  &  Rationale 

Interpret  User 
Needs 

Do  not  have 

defined 

requirements 

Develop 

requirements  from 
lifecycle 
considerations; 
use  prototypes  for 
stakeholder  buy- 
in 

Manage  system 
requirements; 
address  and 
characterize  risk 
associated  with 
requirements; 
conduct  SRR  if 
necessary 

Documented  plan  for 
system  availability, 
supportability, 
logistics  footprint, 
developmental  and 
operational  test 
environments  and 
scenarios,  and 
disposal  in  SEP; 
present  prototype  to 
stakeholders  Sept  05 

Requirements  not 
yet  decomposed; 
RM  started 

Utilized  RM  Tool 

Requirements 
traced  in 
database/tool 

System 

Requirements  Linked 
to  user  Requirements 
in  DOORS  Database 

Refine  System 
Performance 

Specs 

Fundamental 

understand 

performance 

specs 

Documented 

Performance 

Specs 

Refined 

Performance 

Specs 

KPPs  traced  in 
database;  translated 
requirements  into 
performance  specs 
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Calculations 


Jdnteg  rating! 

With  The  Soldier 


•  LOE:  Translate  Value  to  Percent  out  of  100 


Normalized  Gaussian 


y 


H  ±  Ict  =  68.27%  | 


>l±2c?  =  95.45% 
H±3cr  =  99.73% 


Remaining  Work 
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Traceability  &  Budgeting 


•  Traced  to  OSD  &  ARDEC  Guidance 

-  Defense  Acquisition  Guide  “Vee”  Models 

-  Policies,  Process,  Procedures,  Templates 

-  Linked  on  the  SE  Website  for  Ease 

•  Used  to  Develop  SE  Plans  and  Budgets 


Products  That  Radically  Define  Warfare,  Enabling  the  American  Warfighter  to  Dominate  the  Battlefield 
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Key  Areas 


System  Engneering 
Plan 


System  Development  &  Demonstration  :  Pre-Milestone  C 


Defense  AT&L  "V"  Model 


Approved  SEP 


DAG 


ARDEC 


102  ,115 


INPUTS 


All  SE  Activities 


OUTPUTS 


SEP 


Requirements 


Interpret  User  Needs 

4.3.3.3.1 

304 

System  Spec 

<^SRR^> 

Refine  System  Performance  Specs 

4.3.3.3.1 

305-308 

System  ICD 

RTM  to  Functional/Physical  Architectures 

309 

System  OCD 

Environmental  &  Design  Constraints 

310 

Prelim.  Development  Spec 

MOE/MOP 

802 

Prelim  Cl  ICD 

Functional  Analysis 
&  Allocation 


Develop  System  Functional  Specs  &  System 
Verification  Plan 

4. 3.3. 3. 2 

System  Constraints 


Design  Synthesis 


Evolve  Function  Perfo  ace  Spe  ^JIpCI 

Functional  Specs  &  W"' 

III 


Evolve  Cl  Functional 
Documentation  &  lns| 


RAS 

FMEA/FMECA 


ICD 


Fabricate,  Assemble  ^  '■ 

Documentation  H 

4.3. 3.3. 5 

509-510 

IV&V  Plan 

Individual  Cl  Verificatiot^^raE 

4.3.3.8.1 

803-913 

Verification  Procedures 

Integrated  DT&E,  LFT&E,  EOAs  Verify 
Performance  Compliance  to  Specs 

4. 3. 3. 8. 2 

<TRR> 

Verification  & 
Validation 

System  DT&E,  LFT&E,  Oas,  Verify  System 
Functionality  &  Constraints  Compliance  to 
Specs 

4. 3. 3. 8. 3 

Specs,  TEMP,  MOE/MOP,  ICD,  etc. 

Facility  Request 

Staffing  Request 

Data  Request 

Equipment  Request 

Combined  DT&E/OT&E/LFT&E  Demonstrate 
System  to  Specified  User  Needs  & 
Environmental  Constraints 

4. 3. 3. 8. 4 

^<^SVR^> 

DM  Tool(s)  &  Architectures 

111,  115 

Team  with  NWA 

WBS 

202,  205, 

CM  Tool(s)  &  Architectures 

206 

Milestones,  Allotted  Time,  etc. 

Project  Schedule  with  Decision  Points 

System  Analysis  & 
Control 

Track  major  risks  and  execute  risk  strategy 

405 

507-508 

ECP,  CR,  etc. 

CM  Plan 

ICD 

603 

Risk  Analysis  Reports 

Risk  Assessment  Report 

Risk  Mgmt  Plan 

Risk  Status  Report 

Products  That  Radically  Define  Warfare,  Enabling  the  American  Warfighter  to  Dominate  the  Battlefield 
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Traceability  Example 


zl 

ui 

^jnteqratmq  1  ecnjaoiogv^y 
With  The  Soldier 

^  a3^ 


System  Development  &  Demonstration  :  Pre-Milestone  C 


Key  Areas 

Defense  AT&L  "V"  Model 

DAG 

ARDEC 

INPUTS 

OUTPUTS 

Requirements 

Interpret  User  Needs 

4.3.3.1 

304 

System  Spec 

<^SRR^> 

Refine  System  Performance 

Specs 

4.3.3.2 

305-308 

System  ICD 

RTM  to  Functional/Physical 
Architectures 

309 

System  OCD 

Environmental  &  Design  Constraints 

310 

Prelim. 

Development 

Spec 

MOE/MOP 

802 

Prelim  Cl  ICD 

Products  That  Radically  Define  Warfare,  Enabling  the  American  Warfighter  to  Dominate  the  Battlefield 
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SE  Resources  Required 


•  Project  SEWBS 

-  Includes  LOE  Key  Areas 

-  Metrics  to  Obtain  Actual  Data 

•  Top  Down  Method 

-  Step  1 :  Use  Industry  “Rules  of  Thumb”  For 
Initial  Estimate 

-  Step  2:  Refine  Initial  Estimates  Using  the 
LOE  Assessment  Tool 

FY06  SE  Resources  ($)  =  Project  FY06  Budget  ($)  X  Rule  of  Thumb  (%)  X  LOE  (%) 


Products  That  Radically  Define  Warfare,  Enabling  the  American  Warfighter  to  Dominate  the  Battlefield 
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Metric  Tracking  &  Reporting 


•  Tracked  Major  ARDEC  Priority  Project  Database 

-  Status  and  Performance  of  LOE  Key  Areas 

-  Note  Significant  Events  and  Changes 

-  Projects  Evaluated  Monthly  During  Reviews 

•  Reported  at  Senior  Leadership  and  Other 
Management  Reviews  Quarterly 


Products  That  Radically  Define  Warfare,  Enabling  the  American  Warfighter  to  Dominate  the  Battlefield 


16 


pLOPg^; 


Priority  Project  Database 

Snapshot 


El  File  Edit 

@ 


View  Insert  Filter  Tools 
Pose 


Window  Help 

x  v  ©  a  ~  ®  , 


Type  a  question  For 


APO  Org. 
SEL 


Cost  Status 
Change: 

Risk 

Cost  Performance 
Funding 

Schedule  Status 
Change: 

Risk 

Schedule  Performance 
Contracts 
Production 

Performance  Status 
Change: 

Risk 

Performance 
Characteristics 

T  est  and  E  valuation: 

Logistics 
Requirements 

Management 

Interoperability 

IPT  Membership 
IPT  Performance 

Sys  Engrng  Peif 

RM  | 

W  | 

Sys  Engrng  Plan 
Simulation  Support  Plan 


Status  Changed:  [ 
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SE  Status  &  Performance 

Summary 


With  The  Soldier 

a# 


Ih-1 1  Membership 
IPT  Performance 

Sys  Entji  ihj  Port 
RM  | 

W 


FAS 

SA 


Sys  Eii<ii  m<i  Plan 

Simulation  Support  Plan  |_^ 

Status  Changed: 


SE  Status 

— 

i 

i 

i  1 — ■ -  I 

jl\  Date: 
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Reporting  on  Metrics 

SE  Process  STATUS  -  Project  XYZ 

Phase/TRL 


^integrating  i eenaoiog^ ^ 
The  Soldier 


SEL:  Name 

SEP  Status:  (Not  Started, 
Drafted,  Submitted, 

Approved) 

(MM/DD/YYYY) 

Baseline  SE  Level  of  Effort 
(BLOE):  XX%,  (MM/DD/YYYY) 

Previous  SE  Level  of  Effort 
(PLOE):  XX%,  (MM/DD/YYYY) 

Current  SE  Level  of  Effort 
(CLOE):  XX%,  (MM/DD/YYYY) 


Process  Area 

Pert. 

Rationale 

Requirements 

Functional 

Analysis 

Design  Synthesis 

Verification  & 
Validation 

System  Analysis 
&  Control 

Products  That  Radically  Define  Warfare,  Enabling  the  American  Warfighter  to  Dominate  the  Battlefield 
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Benefits 


zl 

ui 

^gdnteg  ratmg  1  eenjaoiogv^y 
With  The  Soldier 

"V  a3^ 


•  Consistent  Documentation  and  Tools  for  Evaluation 

•  Quantified  and  Comparable  Results 

•  Collect  Historical  Data  for  Parametric  Modeling 

•  Provides  Senior  Leadership  Visibility  to  Technical  Issues 
for  ARDEC  Projects 

•  Enforced  Implementation  Through  Reporting 

•  Training  the  Workforce  on  SE 

•  Tailored  to  Provide  Just  Enough  SE;  Avoid  “Process 
Paralysis”  (too  much  SE) 

•  Allows  Project  Manager  to  Focus  on  Important  Issues 

BOTTOM  LINE:  Implementing  Systems  Engineering 
on  Projects  Brings  Better  Products  to  the  Warfighter! 
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Next  Steps 


•  Transition  LOE  from  Pilot  to  Full  Scale 
Implementation 

•  Estimate  SE  Resource  for  FY06  WBS 

•  Track  Status  and  Performance  at  Major  ARDEC 
Project  Reviews  and  Management  Reviews 

•  Gather  and  Incorporate  Voice  of  the  Customer 
Feedback 

•  Refine  and  Improve  LOE  Procedure  and 
Training 


Products  That  Radically  Define  Warfare,  Enabling  the  American  Warfighter  to  Dominate  the  Battlefield 
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Questions/Comments 


Laura  Troiola 

Systems  Engineering  Advisor 

ltroiola@pica.armv.mil 

(973)  724-6296 
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Ensuring  Accomplishment  of  Performance  Based  Logistics  Objectives  Using  Model-Based  Systems  Engineering 
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Department  of  Defense  Instruction  5000.2,  dated  May  12,  2003  set  forth  the  policy  that  logistics  requirements  for  sy 
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inflexible  and  at  times  inappropriate  policies  and  standards.  This  paper  will  demonstrate  how  Model-Based  Systems  E 
used  to  establish,  track,  and  maintain  performance-based  logistics  objectives  and  to  use  system  models  to  predict  lif< 
performance  as  measured  against  the  established  supportability  goals. 
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BILITY  PROGRAM 


Naval  Air  Systems  Command  Integrated 
In-Service  Reliability  Program  (IISRP) 


Mr.  Les  Wetherington,  Program  Manager 

Brief  to  the  NDIA  Systems  Engineering  Conference 

San  Diego,  Ca. 

25  October,  2005 


INTEGRATED  IN-SERVICE  RELIABILITY  PROGRAM 


Agenda 


•  Mission 

•  Vocabulary 

•  Overview 

•  IISRP  Background 

•  IISRP  &  Cost  Wise  Readiness 

•  IISRP  Process 

•  Results 

•  Examples 

•  Summary 


Mission 


SUPPORT  THE  WARFIGHTER  BY 
IMPROVING  RELIABILITY 


“The  nation  needs  a  Navy  that  can  provide  homeland 
defense  and  be  both  forward  and  ready  to  surge  forward 
with  overwhelming  and  decisive  combat  power  ...  As 
leaders,  we  must  create  readiness  from  the  resources 
given  to  us  and  recognize  that  readiness  at  any  cost  is  not 
acceptable.  ” 

ADM  Vern  Clark 
Chief  of  Naval  Operations 

CNO  Guidance  for  2004,  Accelerating  Our  Advantages 


Vocabulary 


•  AERMIP  -  Aircraft  Equipment  Reliability  and 
Maintainability  Program 

•  AMSR  -  Aviation  Maint.  and  Supply  Report 

•  AVDLR  -  Aviation  Depot  Level  Repairable 

•  BCM  -  Beyond  Capability  of  Maintenance 

•  CA  -  Cost  Avoidance 

•  DLA  -  Defense  Logistics  Agency 

•  FST  -  Fleet  Support  Team 

•  IISRP  -  Integrated  In-Service  Reliability  Program 

•  MMH/FH  -  Maint.  Man-Hour  per  Flight  Hour 

•  NAVICP  -  Naval  Inventory  Control  Point 

•  PMA  -  Program  Manager  Air 

•  ROI  -  Return  on  Investment 

•  TOW  -  Time  on  Wing 


Overview 


•  NAVAIR  Integrated  In-Service  Reliability 

Program 

-  A  means  to  sustain  aging  weapon  systems 
components  while  controlling  operations  and 
maintenance  costs 

-An  integral  element  of  NAVAIR’s  global 
strategy  to  meet  the  Chief  of  Naval  Operation’s 
readiness  and  cost  objectives 

•  A  key  component  of  Cost  Wise  Readiness 


INTEGRATED 


IISRP  Background 


AMSR  report  identified  poor  AVDLR  component 
reliability  as  a  major  cost  driver 

NAVAIR  BPR  3-3:  Component  Reliability 
Improvement  Project  initiated  1st  qtr  FY99 

-  AIR-6.0  (Industrial)  leadership,  TYCOMs,  NAVICP, 
AIR-3.0/4.0  (Logistics/Engineering)  participation 

-  Integrated  teams  in  work  at  3  depot  sites  since  1999 

Transitioned  to  an  institutionalized  program  May 
2002 

-  AIR  6. 0/4. 0/3.0  (Industrial/Engineering/Logistics)  Team 


IISRP  &  Cost  Wise  Readiness 


Focus  mainly  on  high  value  AVDLRs: 

-  Identify  poor  performers 

-  Optimize  support  practices 

-  Balance  increased  reliability  vs.  cost 


Objectives 

•Improve  component  reliability 

•  increase  TOW  by  enhancing  fielded  reliability 
•Reduce  Weapon  System  life-cycle  costs 


•  reduce  component  demand,  lower  MMH/FH, 
optimize  O/l/D  capabilities,  increase  readiness 


INTEGRATED 


IISRP  &  Cost  Wise  Readiness 


Involves  all  stakeholders: 


-  Fleet  O-  and  1-Level  Maintainers 

-  PMA/FSTs 

-  Depot  Managers  and  Artisans 


-  NAVI  CP  and  DLA 


•  Every  aspect  of  support  scrutinized 

•  “Fix”  recommendations  linked  to  root  cause 


analysis 

•  Implementation  assistance  and  tracking 


INTEGRATED 


IISRP  &  Cost  Wise  Readiness 


•  Analyzes  components  worked  in  organic 
depots 


-  Primary  focus  on  improving  process 
effectiveness 


-  Achieve  goals  by  maximizing  component 
Time  on  Wing  (TOW) 

-  Ensure  support  processes  restore 
component  resistance  to  failure 


IISRP  Process 


INTEGRATED 


IISRP  Process 


Key  enablers: 

-  Stakeholder  buy-in 

-  Integrated  systems  and  tools 

-  Training  and  expertise 


Select 


Automated  Triggers 


Measure 

Automated 
LCC  Models 


Optimized 

Reliability 


Select 
Top  500  List 
Top  T/M/S  Degraders 


Measure 


BCM/kFH 

TOW 


Analyze 

On-line 
Models  & 
Tools 


Targeted 

Changes 


Reach 

Inherent 

Reliability 


Analyze 
Process  Audits 


Fix 

Follow  the  process 
Consensus  mtng  w/stakeholders 


Cost  Wise  Readiness 
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INTEGRATED 


IISRP  Process 


Select  Analyze  Fix  Measure— I 


PHASE  THREE 

•INSTITUTIONALIZED 
CAPABILITIES 
•PERFORMANCE  BASED 
INDUSTRIAL  FOCUS 
•FORMAL  LIFE  CYCLE 
MODELING 


Capability  pending 
enabling  tools  and 
processes:  SNT, 
depot  data,  etc. 


PHASE  TWO 
•EXPANDED  FOCUS  TO 
DESIGN  /  PERFORMANCE 
•EXPANDED  KNOWLEDGE 
OF  FAILURE  MODE  / _ 


MECHANISM  Where  we  are.. . 


Capability  to 
partially  perform 
with  high  manual 
effort 


•BEGIN  FORMAL  MODELING 


o 

d) 


Automated  trigger 
tools  using  SNTS 

(w/failure  modes  Formal  statistical 
and  depot  data)  reliability 

modeling  tools: 
Weibull,  NHPP, 
LMDSS/  Laplace 

CMIS 


Design/operation 
change  based  on 
complete 

reliability  analysis 


Automated 
LCC/  reliability 
measurements 
using  predictive 
techniques 


analysis 

3M/NALDA 
analysis/SRC 
w/manual  links 
to  failure  modes 


FMEA/FTAs 
(depends  on 
program) 
Rogue  Analysis 


Design/operation 
change  based 
on  partial  data 


Manually 


PHASE  ONE 

•TARGET  TOP  COST  DRIVERS 
•REACH  INHERENT 
RELIABILITY 
•INDUSTRIAL  PROCESS 
FOCUS 


Capability 
exists  to  g 
perform  fully  © 


Summary  listings 
(AMSR/Top  10s) 

Process 
walk  through 

Informal 
discussion  with 
depot/fleet 


Process 

change 


combined 

reports 


Adherence  to 
proper  procedure 


Results  as  of  3rd  Qtr  FY05 


•Delivered  257  Reliability  Studies 

•110  Components  studied  from  the  AFAST  Top 
500  AVDLR  Cost  Driver  List 
•Other  Sources  include 

AMSR,  01,  FST,  IWST,  others 


•Generated  1 383  Actions 


Action  Funding 


Total  #  Funded 
# 


Internal  to  Depot  1307  1292 


External  to  Depot  70  53 


*Combined  6  6 


TOTALS  1383  1351 


=  r  i 


*Combined  =  Actions  with  both  Internal  and  External  requirements. 
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Studies  by  Platform 

Total  Number  of  Studies:  257 


C-130 

0.39% 


AV8 
/  (13) 

5.06% 


Common 

(16) 

6.23% 


F18 

(68) 

26.46% 


EA6B 

(24) 

9.34% 


INTEGS^TIS©  E 


Actions  By  Category 


Packaging  /  Preservation  / 
Handling  /  Shipping  & 
Transportation 
(88) 

6% 


Quality 

(49) 

4% 


Safety 

(8) 

1% 


Tools  and  Support 
Equipment 
(111) 

8% 


Total  Number  of  Studies:  257 


Other 

(36) 

3% 


Material 

(200) 

14% 


Documentation/Processing 

(805) 

58% 


Manpower/Training 

(58) 

4% 


Facilities 

(28) 

2% 


Improvement  Takes  Time 


Effective  Reliability  Investments  Reverse  or  Slow  Cost  Growth..  Over  Time 


BCM  Rate 


INTEGRATED 


Measuring  Results 


Year 

It  takes  time  to  see  initial  results 
ROI  grows  over  time 


-O-fe 


INTEGRATED  E 


Turning  The  Tide 


Examples 


•  The  following  studies  were  completed  by 
local  IISRP  Teams  at  the  Naval  Air  Depots 


•  These  IISRP  Teams  coordinated  with  local 
FSTs,  Fleet  Maintainers,  Depot  production 
managers,  and  artisans  to  complete  the 
analyses 


•  Drivers: 


F/A-18  Horizontal  Servo-cylinder 


-  Ranked  number  20  on  AMSR  List  of  Top  100  AVDLR  Cost  Drivers 

-  High  on  NAVICP  350/360  and  Opportunity  Index  Reports 

-  In  CY98,  922  BCMs 

-  From  1994  to  1999,  BCM/kFH  rate  increased  486% 

•  Findings/Actions: 

-  Majority  of  D-level  repairs  involve  leaking/replacing  seals 

•  Developed  engineering  change  to  replace  dynamic  seals 

•  Issued  LES  directing  100%  replacement  of  seals  in  manifold  and  valve 
assembly  if  compromised  seals  or  rings  are  discovered 

•  Reactivated  Hydraulic  Action  Team  to  train  Fleet  and  reduce  unnecessary 
removals 

-  On  Servo-cylinders  inducted  into  depot,  50%  of  the  Electro-Hydraulic 
Servo  Valves  had  failed 

•  LES  issued  requiring  100%  inspection  of  EHSV  Shuttle  Spool 

•  Implemented  heating  and  cooling  cycling  during  testing 


F/A-18  Horizontal  Servo-cylinder 


Results/Impact: 


-  BCM/kFH  rate  decreased  by  21%  from  existing 
trend  since  3Q  FYOO 

-  Additional  BCM  reduction  expected  after  new 
seals  are  installed 


ELI  ABILITY  PROGRAM 


F/A-18  Horizontal  Servo-cylinder 
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IABILITY  PROGRAM 


F/A-18  Horizontal  Servo-cylinder 
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P-3  Engine  Driven  Compressor 


•  Driver(s): 

-  Ranked  number  30  on  the  AMSR  degrader  list 

-  In  FY99  there  were  141  EDO  BCMs 

•  Findings/Actions: 

-  Findings: 

*  SM&R  code  in  the  O-level  pubs  was  incorrect  and  did  not 
reflect  the  maintenance  plan 

-  Action: 

*  FST  issued  guidance  to  fleet  to  send  EDO’s  to  specialized 
Intermediate  Maintenance  locations 
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P-3  Engine  Driven  Compressor 


•  Results/Benefits: 

-  BCM/kFH  rate  decreased  by  40%  from  existing 
trend  since  IQ  FY01 

-  TOW  increased  by  over  50%  from  existing  trend 
since  4Q  FY02 
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P-3  Engine  Driven  Compressor 
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AV-8B  Stab  Servo-cylinder 


•  Drivers: 

-  First  prototype  IISRP  candidate 

-  In  CY98,  114  BCMs 

-  From  1994  to  1999,  BCM/kFH  rate  increased  215% 


*  Findings/Actions: 

-  Initially,  majority  of  D-level  repairs  involve  leaking/replacing  seals 


•  MCR  released  identifying  wedge-pack  seals  from  Shamban  Aerospace  as 
preferable  substitute.  Total  of  8  seals  per  units  were  impacted 


-  “A/C”  pickoff  testing  procedures  were  inaccurate 

*  Procedures  corrected  and  26  AWP  units  were  retested,  made  RFI 
and  placed  back  into  supply 

-  Sustainment  review  revealed  new  failure  mode:  SAAHS-6  failures 
(electrical) 

•  IISRP  sponsored  OEM  site  visit,  which  revealed  modifications  not 


being  performed  at  depot  level.  Noted  modification  addressed 
electrical  discrepancies 


AV-8B  Stab  Servo-cylinder 


*  Results/Impact: 

-  Resolved  immediate  readiness  issue 

-  Avoided  a  planned  buy  of  new  servo-cylinders 

-  BCM//kFH  rate  decreased  by  55%  from  existing 
trend  since  2Q  FYOO 
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Summary 


•  IISRP 

-  is  a  key  element  of  Cost  Wise  Readiness 

-  is  a  credible  process 

-  has  demonstrated  results: 

*  BCM  Rates  -  reducing  or  slowing  the  increase 

*  TOW  -  improving  or  holding  steady 

-  continues  to  work  with  all  stakeholders  to  improve 
readiness  and  control  cost 
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Immediate  Impact: 

Near  immediate  arrestment  in 
increasing  BCM  trend 
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Long  Term  Results/Benefits: 

S  BCM/kFH  rate  decreased  by 

34%  from  existing  trend  since 

3rd  quarter  FY01 

S  TOW  increased  by  44%  from 
existing  trend  since  3rd  quarter 
FY01 
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Outline  of  Talk 


•  Purpose 

•  Definition  of  systems  engineering  terms 

-  Traditional  Systems  Engineering  (TSE) 

-  Enterprise  Systems  Engineering  (ESE) 

-  Complex-System  Engineering  (CSE) 

•  Characterizing  enterprise  environments 

•  A  regimen  for  CSE 

-  Explanation  of  activities 

-  Preliminary  evaluations 

•  Summary 
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Context  of  This  Talk 


After  [Gharajedaghi,  1999,  p.  31] 


MITRE 


Systems  and  Enterprises  Are  Nested  -  and  see  Notes  page  4 
Changing  Their  Boundaries  Can  Be  Illuminating 


Every  system  or  enterprise  is  part 
of  a  larger  system  or  enterprise. 


Defining  the  boundary  of  a 
system  or  enterprise  is  not  easy. 


Every  system  or  enterprise  has  a 
sub-system  or  sub-enterprise. 


Some  feel  that  no  matter  at  what  scale  one  is,  in  this  nested  structure, 
the  same  known  SE  techniques  can  be  applied  to  effect  good  results. 


Others  say,  no,  depending  on  the  scale  in  question, 
some  radically  different  SE  techniques  may  be  needed. 
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Notional  View  of  Applicability 


ofTSE  and  CSE 


Just  as  some  believe  that  traditional  system  engineering  can  be 
successfully  applied  to  every  system,  there  will  be  those  who 
believe  that  complex-system  engineering  is  appropriate  for  every  system. 


System 

Complexity 


Source:  Mike  Kuras 
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Motivation 

•  Of  course,  there  is  a  continuum  in  thinking  about  this. 

-  There’s  a  whole  spectrum  of  individuals  between  those  taking  a 
traditionalist  view  and  those  searching  for  new  ways  of  systems 
thinking. 

•  We  think  it  is  important  to  offer  a  different  mindset  (the 
regimen)  to 

-  “Capture  the  imagination”  of  those  open  to  it 

-  Provide  “food  for  thought”  for  those  wedded  to  more 
conventional  views. 

•  During  the  following  it  may  help  to  become  a  little  more 
humble 

-  Reverse  (or  suspend)  the  assumption*  that  one  can  always  pre¬ 
specify,  predict,  and  control  system  or  enterprise  behavior  and 
performance 

-  Broaden  your  definition  of  systems  engineering  to  include  the 
management  of  “complex”  environments  that  include  people, 
organizations,  etc. 


*  [Johansson,  2004,  pp.  53-57] 
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System:  An  instance  of  a  set  of  degrees  of  freedom*  having 
relationships  with  one  another  sufficiently  cohesive  to  distinguish 

the  system  from  its  environment. 


** 


Normally  grouped  into  subsets  or  elements  This  cohesion  is  also  called  system  identity 


Ant  Colony 


Hurricane 
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Evolving 


[Kuras  and  White,  INCOSE,  2005] 
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Distinguishing  Attributes  of  Two  Classes  of  Systems 


Complex-systems 

Non-complex  systems 

Unique 

Identical  and  reproducible 

Development  and  operation 
concurrent  and  continuous 

Development  and  operations 
are  separate  and  distinct 

Emergence:  development  and 
operation  at  multiple  scales 

One  predominant  scale 
amenable  to  reductionist 
analysis  and  synthesis 

Stochastic,  unpredictable 

Predictable  at  its  predominant 
scale 

Always  open 

Treatable  as  closed  or  with 
completely  specified  inputs 

Learning  and  memory  of  prior 
history  alters  behavior 

Repeatable  transients 

Requires  both  cooperation  and 
competition  to  function 
effectively 

Competition  (for  resources), 
friction  and  so  forth  reduce 
effectiveness 

[Kuras,  2005] 
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Distinguishing  Attributes  of  Two  Classes  of  Systems  (Concluded) 


Complex-systems 

Non-complex  systems 

Robust  and  broadly  inefficient 

Can  be  optimized  and  made 
efficient 

Ambiguous  and  shifting 
boundaries 

Well-defined,  distinct 
boundaries  at  its  predominant 
scale 

Explores  and  tests  new 
possibilities 

Development  progressively 
removes  unwanted  possibilities 

Self-integrating  and  re¬ 
integrating 

Integrated  by  external  agents  in 
one  or  more  configurations 

Dominated  by  transient  and 
short-range  relationships 

Dominated  by  uniform  and 
permanent  relationships 

Can  exhibit  relational  networks 
at  O(n),  0(n2),  and  0(~2n) 

Can  exhibit  relational  networks 
at  O(n)  and  0(n2) 

Hierarchies  are  partial  and 
transient 

Hierarchies  are  important, 
extensive,  and  durable 

[Kuras,  2005] 


Assertion:  Complex-systems  can  only  be  engineered  by 
intervention,  not  by  specification  and  then  development. 
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Complex-Systems  and  CSE  vs. 
Non-Complex  Systems  and  TSE 


•  Complex-systems  evolve  naturally 

-  Non-complex  systems  do  not. 

•  Many  organizations  are  complex-system  enterprises. 

(see  next  chart) 

•  CSE  creates/shapes  environmental  conditions  which  focus 
and  accelerate  actions  of  people/organizations. 

•  CSE  is  complementary  to  TSE. 

•  TSE  is  applicable  to  some  of  the  parts  of  an  enterprise. 

-  TSE  techniques  should  still  be  applied  when  appropriate. 

-  TSE  is  not  to  be  abandoned. 


[Kuras  and  White,  MIT,  2005] 
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Enterprises 

•  Enterprises  are  complex-systems  functioning  at  multiple  scales. 

-  Scale:  Combination  of  {field  of  view,  resolution}  plus 

{organizational,  process,  technical}  aspects 

-  Often  “emergence’  occurs  &  “patterns”  appear  when  changing  scales. 

•  Enterprises  are  characterized  by  homeostatic*  environments. 

•  Enterprise  evolution  is  driven  primarily  by  people/organizations 
acting  autonomously  but  collectively. 

•  It  is  important  and  useful  to  characterize  the  enterprise’s 
operational  and  developmental  environment. 


*  [Yates,  2002] 


[Kuras  and  White,  MIT,  2005] 
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ESE  Environment  Characterization  Template 


System 

Context 


Strategic 

Context 


Stakeholder 

Context 


piemen  ta  tion 
Context 


•Typical  program  domain 

-  Traditional  systems  engineering 

-  Chief  Engineer  inside  the 
program;  reports  to  program 
manager 

•Transitional  domain 

-  Systems  engineering  across 
boundaries 

-  Work  across  system/program 
boundaries 

-  Influence  vs  authority 

•  Messy  frontier 

-  Political  engineering  (power, 
control...) 

-  High  risk,  potentially  high  reward 

-  Foster  cooperative  behavior 


Source:  Renee  Stevens 
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Regimen  for  CSE 

•  A  regimen  (not  recipe)  for  CSE 

-  Developed  by  SEPO’s  Mike  Kuras 

-  In  paper  presented  at  INCOSE’s  2005  Symposium  [Kuras- 
White,  2005] 

•  8  CSE  activities  are  advocated 

-  Emphasize  the  Developmental  Environment. 

-  Shape  Development  During  Operations. 

-  Identify  Outcome  Spaces. 

-  Establish  Rewards  (and  Penalties). 

-  Judge  Actual  Results. 

-  Apply  Developmental  Stimulants. 

-  Characterize  Continuously. 

-  Enforce  Safety  Regulations. 

•  The  above  activities  are  not  independent  of  one  another. 
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Emphasize  the  Developmental  Environment 

•  Define,  augment,  and  shape  enterprise  environment  to  be 

-  Conducive  to  change/evolution 

-  Supportive  of  both  cooperation  and  competition. 

•  Don’t  try  to  “build”  the  complex-system;  it  builds  itself. 

-  Heed  “the  gardener”  (not  “the  watchmaker”)  metaphor. 

•  If  it  doesn’t  rain... 

•  If  rabbits  are  eating  the  plants... 

-  Understand  “the  shopping  mall”  metaphor. 

•  Methods  for  engineering  environments  are  inherently  open 
ended,  e.g., 

-  Modulate  the  flux  of  developers,  e.g., 

•  Establish  stipends  for  participation 

•  Ensure  unfettered  information  exchange 

-  Manage  towards  stability  in  the  face  of  changes  like  people 
joining  or  leaving  the  environment. 

-  Divert  funds  from  contract  awards  to  performance  rewards. 

-  Use  both  in  situ  environments  and  partially  artificial  extensions. 
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Shape  Development  During  Operations 


Development  and  operation  overlap 
and  occur  simultaneously  in  a 
complex  system.  The  life  cycle  is 
not  development  and  then 
operations. 

Engineering  should  be  applied  to 
operations  as  well  as  to 
development. 

Interoperability  at  different  scales 
requires  different  mechanisms. 

Provide  mechanisms  for 
developmental  collaboration  across 
the  enterprise. 

Examples 

-  Involve  operators  in  development 
(JEFX,  JWID,  ADOCS,  etc.) 

-  Involve  developers  in  operations 
(Joint  STARS  in  ‘91) 


Developmental 
collaboration, '  \ 
mechanisms  are 
focused  here. 
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Things  like  Service  Oriented 
Architectures  (So As)  are  focused  here. 
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Identify  Outcome  Spaces 

Identify  and  formulate  broad  Outcome  Spaces  that  appeal  to  many 
enterprise  participants,  not  narrow  and  specific  outcomes. 

Focus  and  shape  evolution  while  focusing  on  goals;  do  not  try  to  pre¬ 
specify  an  end-state. 

-  Operational  Outcome  Spaces  do  not  always  directly  inform  development. 

-  Developmental  Outcome  Spaces  do  not  directly  determine  operations. 

-  If  specific  desired  outcomes  can  be  achieved  directly  by  individual  entities, 
then  encourage  competition. 

-  If  collective  action  is  required  to  achieve  outcomes,  then  encourage 
cooperation. 

Examples  of  “good”  Outcome  Spaces 

-  U.S.  Army’s  “Own  the  Night” 

•  Not:  Detailed  specifications  for  night-vision  goggles 

-  The  “X-Prize” 

•  Take  a  passenger  into  space,  return  to  earth,  and  then  repeat  within  a  week  with 
the  same  method. 

-  2005  DARPA  “Grand  Challenge” 

•  Advance  technologies  that  will  save  the  lives  of  our  uniformed  men  and  women 
on  the  battlefield. 

-  Neutralize  hostile  cruise  and  ballistic  missile  threats  to  the  U.S. 

•  Destroy  any/all  incoming  cruise  and  ballistic  missiles  before  impact.  MITRE 
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Establish  Rewards  (and  Penalties) 


•  It  is  assumed  that  each  autonomous  agent  of  an  enterprise 

-  Makes  decisions  and  takes  actions  to  achieve  what  they  perceive  as 
desired  outcomes 

-  Is  motivated  by  externally  applied  rewards  and  penalties 

•  These  actions  determine  enterprise  change/evolution. 

•  Rewards  should  link  specific  populations  of  operators  and/or 
developers  to  Outcome  Spaces. 

-  Create  financial  and  other  types  of  incentive  opportunities  for  groups  of 
independent  contractors,  not  for  individual  programs. 

•  Rewards 

-  Influence,  but  do  not  specify,  decision  making  outcomes 

-  Can  accelerate  enterprise  change/evolution 

•  Achievement  Rewards  are  not  contract  awards. 

-  Typically  contracts  are  awarded  before  outcomes  are  achieved. 

-  Rewards  are  for  performance  and  not  the  plausibility  of  promises. 


•  Example  of  a  “good”  Reward 

-  $10  million  and  a  plaque  for  the  X-prize 
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Judge  Actual  Results 

Judging  is  the  explicit  assignment  of  Rewards  to  appropriate 
autonomous  agents  for  actual  outcomes  achieved. 

The  Judging  activity  of  the  CSE  regimen 

-  Ties  Rewards  to  actual  outcomes 

-  Provides  opportunities  to  “weed  the  garden” 

-  Completes  Outcome  Space-to-Rewards-to-autonomous  agents  linkage 

-  Is  tightly  coupled  to  Development  Environment  and  Rewards 

When  change  occurs  in  an  enterprise,  the  acceptability  of  the 
change  needs  to  be  determined. 

-  For  example,  change  should  not  inhibit  future  change  and  should  not 
prevent  the  enterprise  from  continuing  to  operate  successfully. 

-  A  “healthy”  enterprise  does  not  become  less  “complex”  as  it  evolves. 

Rewards  for  positive  change  should  be  allocated  to  those 
responsible  for  its  achievement. 

Rewards  modulate  resource  flows  from  the  environment  to  the 
enterprise. 

Examples 

-  X-Prize 

-  DARPA  Grand  Challenges 
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Apply  Developmental  Stimulants 


•  Accelerate  desired  outcomes  by  stimulating  autonomous  agents  to 
interact  appropriately. 

-  “Stir  the  pot”  and/or  “change  the  rules”. 

-  This  is  the  most  significant  factor  in  accelerating  enterprise  evolution. 

•  Outside  agents  may  be  able  to  facilitate  the  necessary  interactions, 
so  inject  additional  autonomous  agents  as  facilitators  and  brokers. 

-  Example:  MITRE  as  facilitator  of  “Cursor  on  Target  (CoT)”. 

•  Autonomous  agents  should  be  making  “informed”  decisions. 

-  Endeavor  to  increase  the  frequency,  intensity,  and  persistence  of 
autonomous  agent  interactions. 

•  Developmental  Stimulants  are  not  outcomes. 

-  They  encourage  autonomous  agents  to  create  outcomes  for  which 
they  are  mutually  and  not  individually  accountable. 

•  Pay  for  collective  results;  for  example 

-  Modify  DD-250  Form  to  Reward  a  working,  integrated  system. 

-  No  autonomous  agent  (contractor  team)  gets  paid  for  delivering  a 
component  system  that  is  not  successfully  integrated. 
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Characterize  Continuously 


Capture  and  publish  current  “features”  of  the  enterprise  and  its 
environment  that  seem  to  matter  (e.g.,  Outcome  Spaces  and 
actual  outcomes  achieved,  Rewards,  and  Judging  results). 


-  Help  autonomous  agents  to  “think  globally  but  to  act  locally”. 

-  Focus  on  “now”  and  do  not  try  to  pre-specify  the  distant  future. 

-  Continuously  refine  these  features  to  gain  consistency  in  agent 
actions. 


-  Ensure  that  accurate  evaluation  criteria  and  metrics  are  developed 
and  publicized  for  refined  levels  of  the  features. 

-  Avoid  too  much  detail  (refinement)  because  metrics  and  efforts  may 
become  localized  and  not  support  overall  enterprise  performance 
improvement. 

-  Balance  the  continuing  characterization  of  existing  features  with 
initiating  the  characterization  of  new  features. 

Analogical  examples 

-  The  daily  stock  market  report 

-  Highway  traffic  reports 

-  Best/most  recent  Time  Critical  Targeting  (TCT)  times 
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Enforce  Safety  Regulations 


Safety  Regulations  focus  on  ensuring  the  continuous  operation 
of  the  complex-system  or  enterprise  -  not  on  what  it  does  or 
does  not  do. 


-  Formulate  and  enforce  rules  that  keep  the  enterprise  functioning. 

-  Develop  and  monitor  measures  of 

.  “Fitness” 


•  Measures  of  the  rate  of  change 

Guard  against  complex-system  failure  modes:  stagnation, 
disintegration,  or  collapse. 

-  Absence  of  change  may  signal  the  potential  death  of  the  enterprise. 

-  Ensure  change  can  occur  without  destabilizing  or  destroying  the 
enterprise. 

Examples 

-  Criteria  for  vetting  or  training  new  autonomous  agents  as  well  as 
“weeding  out”  dysfunctional  ones 

-  Enforcing  contractual  obligations  among  autonomous  agents 

-  Managed  redundancy/retirement 

•  Microsoft’s  File  Manager  and  Explorer 

-  MIT  Lincoln  Laboratory’s  “off-line,  in-line,  on-line” 
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In  Summary,  Who  Does  All  This? 

•  People  have  asked 

-  Who  is  responsible  for  making  all  this  happen?! 

-  Who  actually  “engineers  the  environment”  of  the  enterprise  to 
accelerate  its  evolution? 

•  These  are  good  questions  beyond  the  present  scope. 

•  The  CSE  regimen  is  akin  to  enterprise  “governance”. 

•  This  role  of  exercising  the  regimen  can  be  taken  by  people 
with  respect,  authority,  power,  and  “purposeful  cohesion”. 

•  It  seems  likely  that  this  “governing  body”  would  be  external  to 
the  enterprise. 
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£  Systems  Engineering  Process  Office 

MITRE-Only  18-Feb  CSE  Workshop 

•  Purpose:  Determine  to  what  extent  the  CSE  regimen  applied  to 
programs 

•  Methodology 

-  Program  experts  provided  basic  information  in  advance 

•  Program  profile:  program  name,  objective,  sponsor,  funding,  years 
involved,  type  and  number  of  contractors,  etc. 

•  Ratings  on  positive/negative  impact  of  each  regimen  activity 

-  Two  hours  were  spent  explaining/discussing  the  regimen. 

-  Each  expert  briefed  their  program  for  about  30  minutes,  focusing  on 
“stories”  about  selected  regimen  activities. 

-  The  wrap-up  discussion  summarized  overall  impressions  about 
applicability  of  regimen  to  programs. 

-  Each  expert  revisited  and  revised  their  pre-meeting  ratings  afterwards 
based  on  what  they  learned  during  the  meeting. 

•  Conclusions 

-  The  regimen  applied  (or  could  have  applied)  to  most  programs. 

-  With  few  exceptions,  the  regimen  had  a  positive  impact. 
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MITRE  Programs  Involved 

•  Department  of  Defense  Intelligence  Information  System 

•  National  Airspace  System  Communications  Modernization 

•  Air  Operations  Center  Weapons  System 

•  Americas  Shield  Initiative 

•  United  States  Visitor  and  Immigrant  Status  Indicator 
Technology 

•  Net  Centric  Enterprise  Services 


•  Theater  Battle  Management  Core  System 
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Numerical  Results 


Total  Effect  for  8  Programs 


□  Did  Apply  Score 

H  Might  Have  Applied  Score 

□  Total 


CSE  Regimen  Activity 


Note  on  Individual  Program  [mgact  Value  (whether  positive  or  negative) 

Major  =  9 
Significant  =  3 
Minor  =  1 
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Numerical  Results  (Concluded) 


Total  Effect  for  8  Programs 
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Summary 

•  A  distinct  mind-set  for  approaching  CSE  has  been  offered. 

-  Concentrate  on  engineering  the  whole  enterprise  environment. 

-  Continue  to  apply  traditional  SE  techniques  to  individual  systems. 

•  Terminology  related  to  traditional  and  enterprise  SE  was  gathered. 

-  Definitions  were  crafted  in  an  attempt  to  foster  better  understanding. 

•  A  template  for  characterizing  ESE  environments  was  suggested. 

•  A  CSE  regimen  for  intervening  in  enterprise  environments  to 
achieve  better  outcomes  was  introduced. 

-  Further  work  is  needed  to  improve  and  validate  the  regimen. 
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Definitions 

System  Definitions  Diagram 


Complex  (Adaptive)  System 


System  of  Systems 
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Definitions  (Continued) 

System :  An  interacting  mix  of  elements  forming  a  whole  greater  than  the  sum  of  its  parts. 

Features:  These  elements  may  include  people,  cultures,  organizations,  policies,  services,  techniques,  technologies, 
information/data,  facilities,  products,  procedures,  processes,  and  other  human-made  or  natural)  entities.  The  whole  is 
sufficiently  cohesive  to  have  an  identity  distinct  from  its  environment. 

Note:  In  this  definition  a  system  does  not  necessarily  have  to  be  fully  understood,  have  a  defined  goal/objective,  or  have  to 
be  designed  or  orchestrated  to  perform  an  activity. 

System  of  Systems  (SoS):  A  collection  of  systems  that  functions  to  achieve  a  purpose  not 
generally  achievable  by  the  individual  systems  acting  independently. 

Features:  Each  system  can  operate  independently  and  is  managed  primarily  to  accomplish  its  own  separate  purpose.  A  SoS 
can  be  geographically  distributed,  and  can  exhibit  evolutionary  development  and/or  emergent  behavior. 

Complex  System:  An  open  system  with  continually  cooperating  and  competing  elements. 

Features:  This  type  system  continually  evolves,  changing  its  behavior  in  response  to  itself  and  its  external  environment 
(often  in  unexpected  ways).  Changes  between  states  of  order  and  chaotic  flux  are  possible.  The  relationships  of  the  elements 
are  imperfectly  known,  and  are  difficult  to  describe,  understand,  predict,  manage,  control,  design,  and/or  change. 

Notes:  Here  “open”  means  free,  unobstructed  by  artificial  means,  and  with  unlimited  participation  by  independent  agents 
and  interactions  with  the  system’s  environment.  Also,  a  complex  system  that  is  entirely  natural  is  not  an  enterprise  (see 
below). 

Enterprise:  A  complex  system  exhibiting  a  relatively  stable  equilibrium  among  many 
interdependent  component  systems  in  a  shared  human  endeavor. 

Features:  An  enterprise  may  be  embedded  in  a  more  inclusive  complex  system.  External  dependencies  may  impose 
environmental,  political,  legal,  operational,  economic,  legacy,  technical,  and  other  constraints. 

Notes:  According  to  this  definition,  an  enterprise  need  not  include  an  agreed-to  or  defined  scope/mission  and/or  set  of 
goals/objectives.  In  addition,  there  is  no  attempt  to  include  what  is  necessary  to  embody  a  successful  enterprise;  that  is  a 
different  topic,  i.e.,  enterprise  engineering  and  enterprise  systems  engineering  (see  below). 
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Definitions  (Continued) 

Engineering :  Methodically  conceiving  and  implementing  solutions  to  real  problems,  with 
something  that  is  meant  to  work. 

Note:  This  definition  does  not  imply  that  the  problems  are  always  solved. 

Enterprise  Engineering :  Application  of  engineering  efforts  to  the  enterprise  with  emphasis 
on  enhancing  capabilities  of  the  whole  and  understanding  the  relationships  and  interactive 
effects  among  the  components. 

Note:  This  definition  does  not  necessarily  imply  that  the  “best”  efforts  are  applied.  (See  enterprise  systems  engineering  on 
next  chart.) 

Systems  Engineering :  An  iterative  and  interdisciplinary  management  and  development 
process  that  defines  and  transforms  requirements  into  an  operational  system. 

Features:  Typically,  this  process  involves  environmental,  economic,  political,  and  social  aspects.  Activities  include 
conceiving,  researching,  architecting,  utilizing,  designing,  developing,  fabricating,  producing,  integrating,  testing, 
deploying,  operating,  sustaining,  and  retiring  system  elements. 

Notes:  The  customer  for  or  user  of  the  system  usually  states  the  initial  version  of  the  requirements.  The  systems  engineering 
process  is  used  to  help  better  define  and  refine  these  requirements.  Further,  often  the  requirements  change  as  further 
decisions  are  made  as  a  result  of  systems  engineering.  Hence,  for  conciseness,  the  use  of  the  single  word  “defines”.  This 
definition  does  not  imply  that  a  successful  system  is  always  realized.  The  word  “integrated”  is  not  included  in  this  definition 
because  systems  engineering  efforts  may  not  be  that  well  integrated. 
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Definitions  (Concluded) 


Traditional  Systems  Engineering  (TSE):  Systems  engineering  but  with  limited  attention  to 
the  non-technical  and/or  complex  system  aspects  of  the  system. 

Features:  In  TSE  there  is  emphasis  is  on  the  process  of  selecting  and  synthesizing  the  application  of  the  appropriate 
scientific  and  technical  knowledge  in  order  to  translate  system  requirements  into  a  system  design.  Here  it  is  normally 
assumed  and  assured  that  the  behavior  of  the  system  is  completely  predictable.  Traditional  engineering  [not  just  TSE] 
typically  is  directed  at  the  removal  of  unwanted  possibilities. 

Note:  Here  it  is  assumed  that  TSE  is  identical  to  “classical”  systems  engineering,  i.e.,  customary  and  accepted  methods  of 
doing  system  engineering. 

Enterprise  Systems  Engineering  (ESE):  A  regimen  for  engineering  “successful”  enterprises. 

Features:  ESE  is  systems  engineering  but  with  emphasis  on  that  body  of  knowledge,  tenets,  principles,  and  precepts, 
having  to  do  with  the  analysis,  design,  implementation,  operation,  and  performance  of  an  enterprise.  The  enterprise 
systems  engineer  concentrates  on  the  whole  as  distinct  from  the  parts,  and  its  design,  application,  and  interaction  with  its 
environment.  Some  potentially  detrimental  aspects  of  TSE  are  given  up,  i.e.,  not  applied,  in  ESE. 

Notes:  Here  “regimen”  means  a  prescribed  course  of  engineering  for  the  promotion  of  enterprise  success. 


Comnlex-S vstem  Engineering  (CSE):  ESE  but  with  additional  conscious  attempts  to  further 
open  the  enterprise  to  create  a  less  stable  equilibrium  among  many  interdependent 
component  systems. 


Features:  In  CSE,  special  attention  is  paid  to  emergent  behavior,  especially  due  to  the  openness  quality,  which  can  either 
be  desirable  or  undesirable.  One  tries  to  instill  the  deliberate  and  accelerated  management  of  the  natural  processes  that 
shape  the  development  of  complex  systems. 
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Multiscale  View  of  Complexity 
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From:  Kuras,  M.  L.,  and  B.  E.  White,  “Engineering  Enterprises  Using  Complex-System  Engineering,” 
Paper  for  INCOSE  10-14  July  2005  Symposium,  Rochester,  NY 
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Set  Theory  View  of  Engineering  Disciplines 
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Derived  from  Renee  Stevens’  template  (see  Chart  8) 

These  “rings”  should  be  interpreted  as  “partitioned”  versions  of  the  rings  of  Chart  32,  e.g.,  Engineering'  above  is  that  portion  of 
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Traditional  System  Engineering 


•  Underpinnings  of  classical  linear  • 

system  analysis 

•  Hierarchical  composition  of  separately 
engineered  subsystems  is  common 

•  Addresses  the  form,  fit,  and  function 
of  a  solution  for  a  problem  in  two 
basic  steps 

-  First,  functionality 

-  Then,  implementation 

•  Starts  with  “specifications” 

-  Specifications  are  predictions  that  are 
made  to  come  true. 

-  Systems  are  built  to  “stand  alone”. 

•  Predictions  carry  a  lot  of  weight. 

-  Plans,  roadmaps,  schedules,  etc.  • 

-  Developmental  tests  are  planned 
independently  of  implementation. 

-  When  there  is  divergence,  one  tries  to 
restore  the  validity  of  the  predictions. 


Many  detailed  tools  and  procedures 

-  Requirements  analysis,  allocation,  and 
traceability 

-  Functional  analysis/synthesis,  tradeoffs, 
abstractions,  structuring  and  layering 

-  WBSs,  PERT  and  Gantt  charts,  etc. 

-  Developmental  processes  (waterfall  and 
spiral  models,  etc.),  developmental  and 
quality  metrics,  configuration  control,  etc. 

-  Modeling/simulation,  OSS&E,  C4ISPs,  ICDs 

-  Technology  surveys  and  risk  management 

-  Unit  &  integration  testing,  OT&E,  MTPs,  etc. 

-  System  architecting  (operational  views, 
employment  views,  technology  views, 
materiel  views,  acquisition  views,  etc.) 

Many  techniques  are  applied  and  refined 
successfully  at  the  product  level 

-  Linearize  non-linear  problems  (externalize 
memory;  employ  feedback). 

-  More  detail  is  always  beneficial. 

-  Iterate  when  possible. 

-  Bottom-up  and  top-down  convergence  also 
helps. 


Abridged  from:  Kuras,  M.  L.,  12  Jan  05,  “Introduction  to  Complex-system  Engineering,”  for  the  Air  Force  (AF) 
Scientific  Advisory  Board  (SAB);  Skip  Saunders’  subcommittee  on  System  of  Systems  (SoS)  Study. 
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Driving  SE  Back  Into  Programs 

[Good  Systems  Engineering  Plans  (SEPs)  Are  Key] 

Technical  Reviews  Interactive  Timeline 


Increased  use  of  disciplined  SE,  including  formal  technical 
reviews,  to  effectively  address  all  technical  issues 

From:  “Driving  Systems  Engineering  into  Programs,”  Mark  D.  Schaeffer,  Principal  Deputy  Director, 

Defense  Systems,  Director,  Systems  Engineering,  Office  of  the  Under  Secretary  of  Defense  (AT&L), 

23  March  2005,  Keynote  for  CSER,  Stevens  Institute  of  Technology 


17 
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Abbreviations  and  Acronyms 

ADOCS  =  Air  Defense  Operations  Center  System 

AF  =  Air  Force 

AOC  =  Air  Operations  Center 

ASR  =  Acquisition  Strategy  Report 

AT&L  =  Acquisition,  Technology,  and  Logistics 

C4ISP  =  Command,  Control,  Communications,  Computers,  and  Intelligence  Support  Plan 

CCRP  =  Command  and  Control  Research  Program 

CDR  =  Critical  Design  Review 

CoT  =  Cursor  on  Target 

CSE  =  Complex-System  Engineering  (or  cSE) 

CSER  =  Conference  on  Systems  Engineering  Research 

CTC  =  Concurrent  Technologies  Corporation 

DARPA  =  Defense  Advanced  Research  Projects  Agency 

DoD  =  Department  of  Defense 

DOS  =  Disk  Operating  System 

ESD  =  Engineering  Systems  Division 

ESE  =  Enterprise  Systems  Engineering 

FOC  =  Full  Operational  Capability 

FoV  =  field  of  view 

FRP  =  Full  Rate  Production 

IBR  =  Initial  Baseline  Review 

ICD  =  Interface  Control  Document 

INCOSE  =  International  Council  on  Systems  Engineering 

IOC  =  Interim  Operational  Capability 

IOTE  =  Initial  Operational  Test  &  Evaluation 

ISR  =  Independent  Safety  Review 

IT  =  information  technology 

ITR  =  Independent  Technical  Review 

JEFX  =  Joint  Expeditionary  Force  Experiment 

Joint  STARS  =  Joint  Surveillance  &  Target  Attack  Radar  System 

JPDO  =  Joint  Planning  and  Development  Office 

JWID  =  Joint  Warfighter  Interoperability  Demonstration 

MIT  =  Massachusetts  Institute  of  Technology  MITRE 
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Abbreviations  and  Acronyms  (Concluded) 

MTP  =  Maintenance  Test  Plan  [or  Package] 

NDIA  =  National  Defense  Industrial  Association 
NECSI  =  New  England  Complex  Systems  Institute 
O  = order 

OOS&E  =  Operational  Safety,  Suitability  and  Effectiveness 

OT&E  =  Operational  Test  and  Evaluation 

OTRR  =  Operational  Test  Readiness  Review 

OUSD  =  Office  of  the  Under  Secretary  of  Defense 

PCA  =  Physical  Configuration  Audit 

PDR  =  Preliminary  Design  Review 

SAB  =  Scientific  Advisory  Board 

SE  =  systems  engineering 

SEP  =  Systems  Engineering  Plan 

SEPO  =  Systems  Engineering  Process  Office 

SFR  =  System  Functional  Review 

SoA  =  Service  Oriented  Architecture 

SoS  =  System  of  Systems 

SoSECE  =  System  of  Systems  Engineering  Center  of  Excellence 

SRR  =  System  Requirements  Review 

TCT  =  Time  Critical  Targeting 

TRA  =  Technical  Readiness  Assessment 

TRR  =  Technical  Readiness  Review 

TSE  =  Traditional  Systems  Engineering  (or  System) 
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Speaker  Contact  Information 


Brian  E.  White,  Ph.D. 
Telephone:  (781)  271-8218 
The  MITRE  Corporation 
E-Mail:  bewhite@mitre.org 
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FUTURE  COMSAT  SYSTEMS 


Agenda 

One  Torn-  TheAmyJDefenseftfuSi&tiy 

•  FCS  and  Army  Transformation 

*  Supportability  Performance 

*  Analysis  Process  and  Examples 

•  Process  Enablers 

•  Lessons  Learned 

*  Questions 
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Leading  Transformation 


FUTURE  COMRAT SYSTEMS 


One  Tesm-IfieAtmy/Defease/ifidtiStty 


•  The  US  Army  “At  War  and  Transforming” 

-781,000  to  480,000  active  duty  since  1990 

-Forces  currently  deployed  in  120  countries 

-Army’s  transformation  effort  announced 
in  Oct  1999 

-Leading  implementation  of  network-centric 
operations 

-Driving  Joint  interdependency  and  standards 

#  PCS'  General  Peter  J.  Schoomaker 

'  .  M  u.  ,  n-  -  Chief  of  Staff,  U.S.  Army 

Transformation  in  Multiple  Dimensions 

-Warfighting,  logistics,  technology,  business 


FCS  is  a  Complex  System  of  Systems  in  a 
Transformational  Warfighting  Context 


Approved  for  Public  Release,  Distribution  Unlimited,  TACOM  6  Oct  2005,  Case  05-223 


IWW  7  October  2005  FCS  4 


Reaffirming  the  Government’s 
Key  Program  Tenets 


fUWfti  COMSAT  SYSTEMS 


One  Teem-  IfieAmy/Defeme/Induslry 


•  Create  opportunity  for  Best  of  Industry  to  participate 

•  Leverage  government  Technology  base  to  maximum  extent 

•  Associate  on-going  enabling  efforts  with  LSI-Led  activity 

•  Collaborative  Environment  from  design  through  life  cycle 

•  As  a  minimum,  Commonality  at  subsystem/component  level 

•  Design/plan  for  Technology  Integration  and  Insertion 

•  Maintain  and  shape  the  Industrial  Base  for  the  future 

•  Retain  Competition  throughout  future  force  acquisition 

•  Appropriate  Government  Involvement  in  procurement 
processes 

•  Consistent  and  continuous  Definition  of  Requirements 

•  Maintain  and  shape  government  acquisition  community 

•  Program  Affordability  -  Balance  performance  and  sustainment 

•  One  team  operating  with  Partnership  and  Teamwork 


The  tenets  remain  constant:  Applying  them  to  the  Current  and  Future  Force 
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FUTURE  COMRAT SYSTEMS 


Future  Combat  Systems 


One  Tesm-IteAtmy/DefsassfttidtiSfry 
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$  Unmanned  Ground  Vehicles 

ARvflRSTA  ARV  Aslt 


FCS  Recovery  and 
Maintenance  Vehicle 


Small 
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Armed  Robotic  Vehicle 


MULE:  (Transport) 


Medical  Treatment 
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FUTURE  COMSAT  SYSTEMS 


Agenda 

One  Torn-  TheAmyJDefenseftfuSi&tiy 

•  FCS  and  Army  Transformation 

*  Supportability  Performance 

*  Analysis  Process  and  Examples 

•  Process  Enablers 

•  Lessons  Learned 

*  Questions 
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FUTURE  COMSAT  SYSTEMS 


Supportability  Performance 

One  Tern-  TheAmyJDefenseflndu&tiy 

*  Supportability  Performance  Objectives 

-  Reduced  Logistics  Footprint 

-  Reduced  Demand  for  Maintenance 

-  Reduced  Demand  for  Supply 


•  Enabled  by 

-  Personnel  Efficiencies 

-  Improved  Reliability/Availability 

-  Lower  Maintenance  Ratio 

-  Increase  in  Crew-performed  Maintenance 

-  Lower  Consumption  Rates 

-  Part  and  supply  Commonality 

-  Self-Sustainment 


-  Networked  Sustainment 
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The  Integrated  -  Interoperable  UA 

Network-Centric  Warfighting  -  Supportability 


FUTURE  COMRAT SYSTEMS 


One  Tesm-IteAmy/DefeasaftfidtiStty 


Joint  Common 
Database 


GP32608055d.ppt 
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FUTURE  COMSAT  SYSTEMS 


Supportability  as  a  Quality  of  Firsts  ^SSS 

Om  Tsm-  TheAmy/DefeM&Mn&ky 


•  See  First 

-  The  Networked  Sustainment  system  “sees”  supportability  concerns  before 
the  warfighter 

•  Understand  First 

-  Networked  Sustainment  system  understands  the  impact/influence  of 
supportability  concerns  on  the  force 

•  Act  First 

-  Networked  Sustainment  system  automatically  presents  Courses  of  Action 
(COAs)  to  the  User  to  resolve  supportability  concerns 

-  Automated  initiation  of  COAs 

•  Finish  Decisively 

-  Networked  Sustainment  enables  resolution  of  supportability  concerns  with 
minimal  impact  to  force  operation 

•  Sustainment  Concerns  =  need  for  and  status  of: 

-  Resupply 

-  Maintenance 

-  Combat  Health  Support 

-  Human  Resource  Support 
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FUTURE  COMSAT  SYSTEMS 


Sustainment  Performance  Analysis  ^SSS 

One  Tsm-  TheAmyJDefm&Mu&ky 


•  Integrate  Army  doctrine  for  supportability  functionality 
into  the  FCS  requirements  baseline 

*  Apply  FCS  Networked  Sustainment  concept  to  the 
accomplishment  of  supportability  functions  in  the  UA 


Design  the 
system  to  be 
sustainable  by 
the  force 


A 


Development  of  an  individual  system 
Current  Army  force  structure 


Design  the 
System-of- 
Systems  to 
sustain  the 
force 


Development  of  a  System-of-Systems 
Future  Force  -  Unit  of  Action 
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FUTURE  COMSAT  SYSTEMS 


Agenda 

One  Torn-  TheAmyJDefenseftfuSi&tiy 

•  FCS  and  Army  Transformation 

*  Supportability  Performance 

*  Analysis  Process  and  Examples 

•  Process  Enablers 

•  Lessons  Learned 

*  Questions 
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Requirements  Tree 


FUTURE  COMSAT  SYSTEMS 


One  Tern-  TheAimyfitefense/Iadwtiy 


Analysis  establishes  a  strong  foundation  to  support  requirements  development 
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FUTURE  COMRAT SYSTEMS 


Analysis  Focus 


One  ham-  iteAtmy/DefeftseflMistry 


MB  SRR  SoSFR  IPDR  PDR  CDR 


DRR  MC 
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Requirement  Decomposition  Process 


FUJVRi  COMSAT  SYSTEMS 


Dm  Tesm-IteAtmy/DefsassftfidtiSlty 
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fUTVRZ  COMSAT  SYSTEMS 

Example  -  Human  Resources  Support 

One  Tesm-IteA/my/DefeasefftidtiSty 

Combat  Services  Support  Army  Universal  Task  List 

Human  Resources  Support  .  Provide  Human  Resources  Support 

Manning  the  Force  .  Man  the  Force 

Personnel  Readiness  Management  .  Conduct  Personnel  Readiness  Management 

Replacement  Operations  Management .  Conduct  Replacement  Operations 

Personnel  Accounting 

Provide  Career  Management 

Personnel  Information  Management .  Provide  Personnel  Information  Management 

Manage  DOD/DA  Civilian  Personnel 

Personnel  Services 
Personnel  Support 

Distribute  soldiers  to  subordinate  commands  based  on  documented  manpower 
authorizations  and  the  commander's  priorities.  ART  6.6.1. 1  involves  the  critical 
manning  tasks  of  predict,  resource,  monitor,  assess,  and  adjust. 


•  Upon  prediction  of  a  critical  personnel  manning  vacancy  based  upon  ....  the  FCS  Networked  System  shall  identify 
the  vacancy  to  the  Commander. 

•  Upon  notification  of  a  vacancy  ...  the  FCS  Networked  System  shall  recommend  assignments  to  fill  critical  personnel 
manning  requirements. 

•  The  FCS  Networked  System  shall  prioritize  critical  personnel  manning  data  for  the  Commander's  assessment. 

•  The  FCS  Networked  System  shall  collect  critical  personnel  manning  data  in  accordance  with  AR  220-1. 

•  The  FCS  Networked  System  shall  recommend  adjustment  of  critical  personnel  to  distribute  soldiers  to  subordinate 
UA  commands. 
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Example  -  Dental  Support 


FUWRi  COMSAT  SYSTEMS 


One  Teem-  IbeAmy/Defeme/Induslry 


Combat  Services  SuoDort 

Health  Service  Support 

Functional  Areas 

Medical  Evacuation  &  Regulation 
Hospitalization 

Health  Service  Logistics 

Armv  Universal  Task  List 

Provide  Force  Health  Protection 

Provide  Combat  Casualty  Care 

Provide  Medical  Treatment 

Provide  Hospitalization 

Dental  Services 

Provide  Dental  Services 

Operationa  1  Ca  re 

Operational  Dental  Care 

Emergency  Dental  Care  , 

Emergency  Dental  Care 

Essential  Dental  Care 

Essential  Dental  Care 

Comprehensive  Care 

Comprehensive  Dental  Care 

Veterinary  Support 
Preventive  Medicine 


Provide  Clinical  Laboratory  Services 
Provide  Mental  Health  Treatment 
Provide  Medical  Evacuation 
Provide  Medical  Logistics 
Provide  Casualty  Prevention 


Out  of  Scope 
For 

Unit  of  Action 


Provide  Preventive  Dentistry  Support 
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FUTURE  COMRAT SYSTEMS 


Example  -  Dental  Support  (page  2) 


One  Tesm-IteAtmy/DefsassftfidtiSlty 


Combat  Services  Support 


Army  Universal  Task  List 
Provide  Dental  Services 


Dental  Services 


Operational  Care 

Emergency  Dental  Care 
Essential  Dental  Care 


Operational  Dental  Care 
Emergency  Dental  Care 
Essential  Dental  Care 


Comprehensive  Care 


Comprehensive  Dental  Care 


Prevent  and  treat  dental  disease  and  injury.  ART  6.5. 1.3  includes  providing 
operational  dental  care,  which  consists  of  emergency  dental  care  and  essential 
dental  care,  and  comprehensive  care  which  is  normally  only  performed  in  fixed 
facilities  in  CONUS  or  in  at  least  a  Level  III  facility. 


Provide  Emergency  Dental  Treatment 

*  Collect  Emergency  Dental  data 

*  Communicate  Emergency  Dental  Data  to  MC4 
Provide  Preventive  Dental  Support 

•  Collect  preventive  Dental  data 

•  Communicate  preventive  Dental  Data  to  MC4 


Out  of  Scope  For  Unit  of  Action 
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Analysis  Summary  and  Results 

One  TheAmyJDefenseftfuSi&tiy 

•  Original  Sustainment  requirements  analysis  based  only  on  the  ORD 
resulted  in  approximately  1100  requirements 

•  Incorporation  of  CSS  and  AUTL  field  manuals  into  the  analysis 
process 

•  CSS/AUTL  analysis  clarified  functionality  not  obvious  in  original  ORD 
analysis 

-  Human  Resources 

-  Information  Management 

•  Medical  Support 

•  Resupply 

•  Maintenance 

-  Planning  functions 

•  Resupply 

•  Maintenance 

•  CSS/AUTL  analysis  derived  an  additional  950  SoS  requirements 

-  Represents  1/3  of  the  Sustainment  Requirements  in  the  specification 
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FUTURE  COMSAT  SYSTEMS 


PIT" 

One  TeM-IteA/my/DefsnssfftidtiSty 

*  FCS  and  Army  Transformation 

*  Supportability  Performance 

*  Analysis  Process  and  Examples 

*  Process  Enablers 

*  Lessons  Learned 

*  Questions 


Agenda 
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FUTURE  COMSAT  SYSTEMS 


Key  Factors  to  a  Successful  Analysis  ^SSS 

Om  Tsm-  TheAmy/DefeM&Mn&ky 


*  The  right  mix  of  people  ...  and  personalities 

-  Systems  Engineers 

-  System  Designers 

-  Logisticians 
-Soldiers 

-  Facilitators 

*  Leadership  commitment  to  a  common  set  of  goals 

*  Adequate  planning  and  schedule 

*  Participants  want  to  do  the  job  and  appreciate  the  value 

*  Maintain  tangible  results  in-sight 
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FUTURE  COMSAT  SYSTEMS 


Agenda 

Dm  Tern-  TheAmyJDefemsftndustiy 


•  FCS  and  Army  Transformation 

*  Supportability  Performance 

*  Analysis  Process  and  Examples 

•  Process  Enablers 


*  Lessons  Learned 


•  Questions 
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Lessons  Learned 


FUTVRi  COMSAT  SYSTEMS 


One  Teem-  IfieAmy/Defeme/Induslry 


*  It  pays  off  when  the  time  is  taken  to  do  the  job  right 

*  Indications  the  job  was  done  right 

-  Endures  the  “test  of  time” 

*  Sustainment  analysis  at  the  front  end  of  the  program  as  a 
major  influence 

-  Historically  unusual  for  this  level  of  Sustainment  requirements 
analysis  this  early  in  a  program 

-  Sustainment  requirements  constitutes  -30%  of  System-of-System 
requirements  on  FCS 

*  Culture  change  within  the  Sustainment  community  ... 
bigger  culture  change  outside  the  Sustainment 
community 
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One  TeM-IteA/my/DefsnssfftidtiSty 

*  FCS  and  Army  Transformation 

*  Supportability  Performance 

*  Analysis  Process  and  Examples 

•  Process  Enablers 

•  Lessons  Learned 

•  Questions  I 


Agenda 
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System  Safety  in  Systems  Engineering 
DAU  Continuous  Learning  Module 

NDIA  Systems  Engineering  Conference 

October  26,  2005 

Amanda  Zarecky 
Booz  Allen  Hamilton 
703-604-5468 
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Course  Context  -  Drivers 


■  Increased  DoD  emphasis  on  safety 

-  May  2003  SECDEF  Memo 

■  July  2003  Defense  Safety  Oversight  Council 

•  Joint  Chiefs  of  Staff  &  Undersecretaries  of  the  Services 

•  Eight  Task  Forces 

■  April  2004  Acquisition  and  Technology  Programs  Task  Force 

■  Chair:  Mr.  Mark  Schaeffer,  USD  (AT&L)  Director  of  Systems 
Engineering 

■  Focused  on  improving  System  Safety  implementation 

■  Linked  efforts  to  Systems  Engineering  revitalization  initiatives 

■  23  Sep  04  USD(AT&L)  Memo  "Defense  Acquisition  System 
Safety" 
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Course  Context  -  DoD  Policy 


■  23  May  03  DoDI  5000.2  E7,  Environment,  Safety,  and  Occupational 
Health  (ESOH) 

■  Strategy  for  integrating  ESOH  into  Systems  Engineering 

■  Identification  of  ESOH  risks 

■  Acceptance  of  ESOH  risks  per  "industry  standard  for  system  safety" 

■  NEPA/E.0. 12114  Compliance  Schedule 

■  23  Sep  04  USD  (AT&L)  Defense  Acquisition  System  Safety  memo 

■  Mandates  integration  of  System  Safety  into  Systems  Engineering 
"  Mandates  use  of  MIL-STD-882D 

■  Oct  04  Defense  Acquisition  Guidebook 

■  Chapter  4,  Systems  Engineering 

■  Section  4.4.11,  ESOH:  "industry  standard"  =  MIL-STD-882D 
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Course  Development  Team  Effort 


■  USD  (AT&L)/Systems  Engineering 

■  Col  Warren  Anderson,  Program  Manager 

"  Ann  Marie  Choephel,  Program  Manager  Support 

■  DAU  Course  Developer  contractors:  MTC  &  CTC 

■  Subject  Matter  Experts  from  each  Component  and  DAU 

■  Trish  Huheey,  DUSD(I&E)  (Team  Lead) 

■  Sherman  Forbes,  SAF/AQRE 
-  Ben  Mack,  USMC  (AOT,  Inc.) 

■  George  Murnyak,  US  Army  CHPPM 

■  Paige  Ripani,  DUSD(I&E)  (Booz  Allen  Hamilton) 

■  Amanda  Zarecky,  CNO  N45  (Booz  Allen  Hamilton) 
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Course  Description 


■  Course  developed 

■  In  response  to  need  for  training  depicting  how  System  Safety  fits  into 
the  overall  DoD  Systems  Engineering  process  throughout  a  system’s 
life  cycle 

"  To  teach  the  learning  objectives  and  encourage  active  participation 
and  coordination  between  System  Safety  Engineers  and  Systems 
Engineers 

■  Top  Level  Outcomes 

■  Recognize  the  Defense  Acquisition  policy  and  guidance  on  System 
Safety  in  Systems  Engineering 

■  Recognize  System  Safety  methodology  as  the  Systems  Engineering 
approach  for  eliminating  Environment,  Safety,  and  Occupational 
Health  (ESOH)  hazards  or  minimizing  ESOH  risks  across  the 
system’s  life  cycle 
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Course  Description  (cont) 


■  Target  Audience 

■  Primary:  Systems  Engineers,  Chief  Engineers 

■  Secondary:  Program  Managers,  System  Safety  Engineers 

■  DAU  Systems  Engineering  Elective  -  not  required;  no  pre¬ 
requisites 

■  Counts  towards  80  hours  of  DAWIA  certified  continual 
learning 

■  3  y2  hours  web-based  training 
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Course  Description  (cont) 


■  Built  around  the  Systems  Engineering  (SE)  Process  V-Model 

■  Identifies  System  Safety  activities  supporting  each  of  the 
Systems  Engineering  activities  in  each  phase  of  a  systems 
life  cycle 

■  Enables  Systems  Engineers  and  System  Safety  Engineers  to 
understand  what  to  expect,  what  to  provide,  and  when 

■  Not  intended  to  teach  details  of  System  Safety 

■  Assumes  an  understanding  of  Systems  Engineering 


Course  Outline 


■  System  Safety  Overview 

■  System  Safety  Terminology 

■  Eight  Mandatory  Steps  of  System  Safety 

■  Risk  Assessment 

■  System  Safety  Order  of  Precedence 

■  Typical  System  Safety  Tasks 

■  System  Safety  Throughout  the  System's  Life  Cycle 

■  Module  Summary 
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System  Safety  Overview  -  Explains  MIL-STD-882D  methodology  is  DoD's  SE 
approach  for  eliminating  ESOH  hazards  or  minimizing  ESOH  risks  across  the 
system's  life  cycle 


55)  DAU  Continuous  Learning  Confer 


System  Safety  in  Systems  Engineering 

System  Safety  Overview 


Why  Implement  System  Safety? 


Protects  military  and  civilian  personnel  by 
reducing  hazards/risk  to  personnel  and 
equipment 

Reduces  accidents  proactively 

Improves  warfighting  capability  and  combat 
readiness 

Reduces  total  ownership  costs 

Lowers  the  risk  of  environmental  damage 

Prioritizes  hazards  for  corrective  action 

Reduces  need  for  system  retrofits 

Required  by  DoDI  5000,2  (May  2003)  and 
USD(AT&L)  Memo  (Sep  23,  2004) 
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System  Safety  Terminology  -  Defines  terms  pertinent  to  use  of  system  safety 
in  the  SE  process 


System  Safety  Terminology 


System  Safety  in  Systems  Engineering 

System  Safety  Terminology 


Directions:  Listed  below  are  terms  that  are  relevant  to  system  safety  and  the  systems  engineering 
process,  You  may  already  be  familiar  with  some  of  the  terms.  Please  click  each  term  below  that  is 
unfamiliar  to  you  (or  that  you  would  like  to  review)  to  reveal  its  definitions., 


System  Safety  Terms 

System 

System  Life  Cycle 

Systems  Enaineerina 

System  Safety 

System  Safety  Enaineerina 

Environment.  Safety,  and 

Occupational  Health  -fESOH') 

Proarammatic  Environment. 

Safety,  and  Occupational 

Health  Evaluation  fPESHEl 

Human  Systems  Intearation  (HSH 

Hazard 

Causal  Factor 

Mishap 

Risk 

Mitiaation  Measure 

Residual  Risk 
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Eight  Mandatory  Steps  of  System  Safety  -  Describes  application  of  each  of 
the  steps  in  the  system  safety  process  outlined  in  MIL-STD-882D 


DAU  Continuous  Learning  Center 


System  Safety  in  Systems  Engineering 

Eight  Mandatory  Steps  of  System  Safety 


Lesson  Overview 

IG  February  2000 

SLPEKSEDLMC 

1.  Document  the  system  safety  approach 

MlL-STDJftZC 

1  H-  J  IiHLIiL  h 

2*  identify  hazards 

3.  Assess  risk 

DEPARTMENT  OF  DEFENSE 

4.  identify  risk  mitigation  measures 

STANDARD  PRACTICE  FOR 

SYSTEM  SAFETY 

5.  Reduce  risk  to  acceptable  level 

"  ‘  - 

6.  Verify  risk  reduction 

7.  Review  hazards  and  accept  residual  risk  by 

IHh#  .1 

appropriate  authority 

It.  .  t  JS 

8.  Track  hazards,  their  closures,  and  residual  risk 
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Eight  Mandatory  Steps  of  System  Safety  -  Knowledge  Review 


DAU  Continuous  Learning  Center 


System  Safety  in  Systems  Engineering 

Eight  Mandatory  Steps  of  System  Safety 


Drag-and-Drop  Challenge 


Directions:  The  following  are  the  steps  taken  by  the  (fictitious)  Marauder  Howitzer  Program  Office  team 
to  mitigate  the  risk  of  extreme  temperatures  causing  the  gun  barrel  to  warp,  a  round  to  jam  in  the 
barrel,  followed  by  an  in- bore  explosion  that  severely  injures  or  kills  the  operators  and  destroys  the 
Howitzer.  Arrange  the  activities  in  the  order  that  accurately  reflects  the  system  safety  process,  Then 
click  the  Submit  button,  The  Next  button  will  return  to  the  navigation  bar  when  the  answer  is  correct. 
Click  here  if  you  require  a  text-based  version  of  this  challenge. 


Discover  potential  round  jamming  hazard. 


Install  LRDD  tg  detect  gun  barrel  warping. 


Document  the  system  safety  approach. 


Track  rounds  jamming  in  the  gun  barrel. 


Document  PM  acceptance  of  residual  risk. 


Assign  Initial  risk  category  as  high. 


Verify  residual  risk  following  installation  of  LBDD. 


Identify  alternatives  for  eliminating  hazard  or  reducing  risk. 


Submit 
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Risk  Assessment  -  Provides  a  systematic  process  for  assessing  risk  and 
determining  appropriate  risk  acceptance  authority 


MU  Continuous  Learning  Center 


System  Safety  in  Systems  Engineering 

Risk  Assessment 


Overview  of  Mishap  Assessment,  Cont. 


The  following  pages  provide 
further  details  of  each  of  these 
steps  along  with  the  three  sample 
tables: 

1.  Severity  Categories 

2.  Probability  Levels 

3.  Risk  Assessment  and 
Acceptance  Matrix  (shown 
here) 

MIL- STD-882D  provides  definitions 
for  severity  categories,  probability 
levels,  risk  assessment  values  and 
risk  assessment  categories  that 
will  be  used  unless  otherwise 
documented  by  the  Program 
Office  in  the  system  safety 
approach. 

Components  of  Step  1 


D 


, 


Risk  Assessment  and  Acceptance  Matrix 


PROBABILITY 

LEVEL 

SEVERITY  CATEGORY 

1 

CATASTROPHIC 

H 

CRITICAL 

i 

ilARflINAL 

n 

NEGLIGIBLE 

(At  Frequnnl 

-j  pi 

jm 

13' 

[B)  Prohablo 

2m 

5 «'» 

[C)  Occasional 

4<K) 

0  :nci 

■)  gi'wci 

|D)  Remote 

8<IDI 

■jQTO 

14™ 

jgw 

{E|  Improbable 

12'*' 

15* 

-j  yniiEi 

20IIVE| 

Values 

Category 

Acceptance  Authority 

1s2, 3P*,S 

iCompfiiem  Asquis  ■lon'Ejteculi'^e 

B  -W 

SERIOUSl 

I  Prog  ram  Executive  Officer 

10, 11. 12. 13.14.16.lt.  IT 

MEDIUM 

Iproflrarn  Manager 

IB,  11  20 

LOW 

Program  Manager 
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Risk  Assessment  -  Knowledge  Review 


DAU  Continuous  Learning  Center 


System  Safety  in  Systems  Engineering 

Risk  Assessment 


Risk  Acceptance  Authority,  Cont. 


Directions:  Use  the  Risk 
Assessment  and  Acceptance 
Matrix  to  answer  each  of  the 
following  challenges. 

Challenge:  Who  is  the 
acceptance  authority  if  the 
severity  category  is  marginal  and 
the  probability  level  is  frequent? 
Answer 

Challenge:  Who  is  the 
acceptance  authority  if  the 
severity  category  is  catastrophic 
and  the  probability  level  is 
improbable?  Answer 


Risk  Assessment  and  Acceptance  Matrix 


PROBABILITY 

LEVEL 

SEVERITY  CATEGORY 

1 

CATAarpoptic 

« 

CRITICAL 

■ 

MARGINAL 

tit 

NE5J&IBLE 

(A|  F  requern 

-j  m 

3« 

jm 

13w 

[Bj  ProtaMe 

2m 

gmai 

[C)  Occasion  ul 

0,  jlici 

11'ii'd 

1  gi'vci 

|D)  Remote 

81ID| 

-j  Q  [110] 

14'“) 

{EHmprobable 

12«s> 

15'*' 

20HVH 

Values 

Category  Acceptance  Authority 

U3;4,5 

K Ml 

Conpcier:  Asquis  non  E.>.scj?irt 

6JA1 

program  Executive  Officer 

10, 11.12.13.14,15, 15,17 

Program  Manager 

IB.  19.  2D 

LOW 

Program  Manager 

System  Safety  Order  of  Precedence  -  Identifies  and  explains  application  of 
DoD's  system  safety  order  of  precedence  for  eliminating  ESOH  hazards  or 
minimizing  ESOH  risks 


nun  „  .  .  .  „  System  Safety  in  Systems  Engineering 

w W  CorttBOTOJ  letnung Center  Sysfem  Safety  0rder  of  Precedence 


System  Safety  Order  of  Precedence 


When  comparing  potential  alternatives  for  eliminating  the  hazard  or  reducing  the  risk,  the  system 
developer  should  apply  the  MIL-STD-882D  system  safety  design  order  of  precedence,  The  following  are 
listed  in  order  from  the  most  to  the  least  preferred  risk  mitigation  methods: 


Most  to  Least  Preferred  Risk  Mitigation  Measures 

1. 

Eliminate  hazards 

throuah  desian 

selection 

If  unable  to  eliminate  an  identified  hazard,  reduce  the  associated  risk  to  an 
acceptable  level  through  design  selection. 

2. 

Incorporate  safetv 

devices 

If  unable  to  eliminate  the  hazard  through  design  selection,  reduce  the  risk  to 
an  acceptable  level  using  protective  safety  features  or  devices. 

3. 

Provide  warnina 

devices 

If  safety  devices  do  not  adequately  lower  the  risk  of  the  hazard,  include  a 
detection  and  warning  system  to  alert  personnel  to  the  particular  hazard. 

4. 

Develop  procedures 

and  trainina 

Where  it  is  impractical  to  eliminate  hazards  through  design  selection  or  to 
reduce  the  associated  risk  to  an  acceptable  level  with  safety  and  warning 
devices,  incorporate  special  procedures  and  training.  Procedures  may  include 
the  use  of  personal  protective  equipment. 

Note:  For  catastrophic  or  critical  hazards,  avoid  using  warning,  caution,  or 
other  written  advisory  as  the  only  risk  reduction  method. 
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System  Safety  Order  of  Precedence  (cont) 


Marauder  Howitzer  SHA  EXAMPLE  ONLY 

Risk  Mitigation  Measure  lb 


Example  -  Marauder  Howitzer  System  Hazard!  Analysis  Worksheet  -  Risk  Mitigation  Measure  1b 

Hazard 

Hazardous 

Effects 

Causal  Factors 

IS 

IP 

IRV 

IRC 

Risk  Mitigation 

FS 

FP 

FRV 

FRC 

Status 

Round 
jams  in 
barrel 
when 
fired 

Round 
initiates 
causing  m- 
bore 

explosion 
resulting  in 
personnel 
death  and 
weapon 
destruction 

Warped  gun  barrel 
from  a  combination 
of  extreme  external 
temperature,  e.g,, 
in  Desert  Warfare, 
and  high  fine  rate 

1 

B 

2 

High 

Develop  new  barrel 
design  using  new 
technology  composite 
material  that  will 
contain  blast  over 
pressure.  New  barrel 
design  will  minimize 
warping  and  is  a  line 
replaceable  unit  that 
costs  $50K  to 
minimize  downtime  in 
the  event  of  an 
in-bore  explosion. 

This  design  change 
allows  only  minor 
system  damage  and 
no  injury  to  personnel. 

III 

D 

14 

Medium 

Closed.  The  Program 
verified  that  new  barrel 
design  using  the  new 
technology  composite 
material  reduced  the  prob¬ 
ability  of  warping  (causal 
'actor)  and  reduced  the 
seventy  of  the  mishap 
occurring  by  being  able  to 
contain  and  dissipate  the 
blast  overpressure.  The 
Program  Manager  formally 
accepted  the  FRC. 

IS  =  Initial  Risk  Severity  Category 
FS  -  Final  Risk  Severity  Category 
CAE  =  Component  Acquisition  Executive 


IP  =  Initial  Risk  Probability  Level 
FP  =  Final  Risk  Probability  Level 
PEG  =  Program  Executive  Officer 


IRV  =  Initial  Risk  Vblue 
FRV  =  Final  Risk  Value 
PM  =  Program  Manager 


IRC  =  Initial  Risk  Category 
FRC  -  Final  Risk  Category 


D 


Class 


System  Safety  Order  of  Precedence  (cont) 


DAU  Continuous  Learning  Center 


System  Safety  in  Systems  Engineering 

System  Safety  Order  of  Precedence 


Marauder  Howitzer  Risk  Matrix 
Risk  Mitigation  Measure  lb 


EXAMPLE  ONLY 


Risk  Mitigation  Measures  (RMM)  Examples 


PROBABILITY 

LEVEL 

SEVERITY  CATEGORY 

i 

CATASTROPHIC 

91 

CRITICAL 

III 

MARGINAL 

IV 

NEGLIGIBLE 

(A)  Frequent 

1 

3 

7 

13 

{B}  Probable 

2  A. 

5 

9 

16 

(C)  Occasional 

4 

nL 

11 

18 

(D)  Remote 

8 

ioN 

19 

{E}  Improbable 

12 

15 

17 

20 

High  Risk  I  I  Medium  Risk  A  initial  Risk 

Serious  Risk  I  I  Low  Risk  A  Residual  Risk 
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System  Safety  Order  of  Precedence  -  Knowledge  Review 


BJ  dau  Continuous  Learning  Center 


System  Safety  in  Systems  Engineering 

System  Safety  Order  of  Precedence 


Drag-and-Drop  Challenge 


You  are  leading  the  Program  Office  team  of  the  (fictitious)  Marauder  Howitzer.  Please  click  round 
jamming  hazard  to  review  critical  background  information  before  completing  the  following  challenge. 

Directions:  Listed  below  are  four  alternative  measures  to  mitigate  the  round  jamming  hazard.  Based  on 
the  system  safety  design  order  of  precedence,  indicate  the  order  in  which  each  alternative  should  be 
applied  by  placing  the  Number  1  beside  the  most  preferred;  the  Number  2  beside  the  second  preferred; 
etc.  Then  click  the  Submit  button,  The  Next  button  will  return  to  the  navigation  bar  when  the  answer  is 
correct.  Click  here  if  you  require  a  text-based  version  of  this  challenge, 


1 

Train  operators  to  routinely  check  For  barrel  warping. 

2 

Install  alarm  to  alert  operators  to  gun  barrel  warping. 

3 

Install  safety  interlock  that  prevents  Firing  if  the  barrel  is  warped. 

4 

Change  the  design  to  preclude  a  round  jamming  in  the  barrel. 

Submit 
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Typical  System  Safety  Tasks  -  Provides  detailed  descriptions  of  several 
widely-used  system  safety  analytical  and  assessment  tools 


DAU  Continuous  Learning  Center 


System  Safety  in  Systems  Engineering 

Typical  System  Safety  Tasks 


Typical  System  Safety  Tasks,  Cont. 


Typical  System  Safety  Tasks 

Safety  Requirements/Criteria 

Analysis  fSRCAl 

Health  Hazard  Assessment  fHHA) 

Safety  Assessment 

Report  { S  AR} 

Preliminary  Hazard  List  fPHL'j 

Preliminary  Hazard  Analysis  "PHA) 

Subsystem  Hazard 

Analysis  fSSHA’) 

System  Hazard  Analysis  fSHA’) 

ODeratina  &  Support  Hazard  Analysis 

(O&SHAl' 

Sneak  Circuit 

Analysis  fSCA'i 

Fault  Tree  Analysis  ■fFTAl 

Failure  Modes  and  Effects  Analysis  fFMEA') 

Failure  Modes,  Effects,  and  Criticality 

Operational  Trend 

Analysis 

Analysis  fFMECAl 

Threat  Hazard  Assessment 

(THA) 

System  Safety  Proaram  Plan  fSSPPl 
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System  Safety  Throughout  the  System's  Life  Cycle  -  Provides  an  overview 
of  key  system  safety  activities  completed  during  each  phase  of  the  system  life 
cycle 


Mdau  Continuous  Learning  Center 


System  Safety  in  Systems  Engineering 

System  Safety  Throughout  the  System's  Life  Cycle 


System  Safety  Activities  Throughout  the  System  Life  Cycle 


Directions:  Please  click  each  of  the  five  phases  of  the  System  Life  Cycle  to  discover  key  safety 
activities  completed  by  the  system  safety  staff  during  that  phase,  After  you  have  clicked  each  of  the 
five  phases,  please  click  the  Next  button  to  continue, 


Phases 


Work 

Efforts 


L 

A  1 

i  Ask  IOC  FOG 

Concept 

Refinement 

A  Concept 
▼  Dewion 

Technology 

Development 

System  Development  & 
Demonstration 

System  System 

integration  Demonstration 

A  &eaign 

W  Readiness 
*  R^lew 

Production  & 
Deployment 

Fiji  Rate 

lrip  Piodudkiii 

IOTE  iFRP 

■  ['EL'  Flu*-! 

"  Review 

Operations  & 
Support 

Sustainment 

Disposal 

Activities 

Technical 


Pra  Systems  Acquisition  -►-4- 
TRA 

T 


Systems  Acquisition 
TRA 

I 


-+-4—  Sustainment 


Reviews  - *A  A - A  A 

ITR  ASR  SRR  IBR  SFRPDRCDR 

(May  be-  Iffy  l» 
itoie  more- 
than  one)  than  one' 

▼  Technology  Readiness  Assessment  A  Technical  Reviews 


A 

TRR 


*A 

m 

m 


-A 

OTRFt 


A 

PCA 


ISR 


L  \  Program  Reviews 
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System  Safety  Throughout  the  System's  Life  Cycle  (cont) 
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System  Safety  in  Systems  Engineering 

System  Safety  Throughout  the  System's  Life  Cycle 


Concept  Refinement  SE  Chart 


5E  Chari:  Directions  Alternate  document  containing  all  the  information  in  the  chart  below. 


D 


INPUTS 


OUTPUTS 


-  CD 

-  teh  Flan 
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C  lick  each  box  to  read  the  details 
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System  Safety  Throughout  the  System's  Life  Cycle  (cont) 
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System  Safety  Throughout  the  System's  Life  Cycle 


Concept  Refinement  SE  Chart 


SE  Chart  Directions  Alternate  document  containing  all  the  information  in  the  chart  below.  D 


X  Close  Window 


Inputs 

System  Safety  Should: 

Initial  Capabilities  Document  (ICD) 

Provide  inputs  as  requested 

Analysis  of  Alternatives  (AoA)  Plan 

Participate  in  AoA  development 

Exit  Criteria 

Provide  the  following  exit  entena: 

1.  Fuel immaiy  Hazard  List  (PUL) 

2.  Strategy  for  integrating  Environment,  Safety,  and 
Occupational  Health  (ESOH)  nsk  management  into  Ihe 
Systems  Engineering  Plan  {SEP) 

Alternative  Maintenance  and  Logistics  Concepts 

Provide  inputs  as  requested 
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LIvjh  q:  L~!i:iii|i-:rH-W  L-! cn i .  h 
FiimIi  iu-n.,r:  rti>i  Tm:'rrJn:j  h:-. 


Click  cadi  box  to  read  the  details 


QUICK  REFERENCES 


PRINT 


5a  of  15 


4  BACK  NEXT  ► 


System  Safety  Throughout  the  System's  Life  Cycle  -  Knowledge  Review 
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Drag-and-Drop  Challenge 


Directions:  Drag  each  of  the  system  safety  activities  to  the  corresponding  phase  of  the  System  Life 
Cycle.  Then  click  the  Submit  button.  The  Next  button  will  return  to  the  navigation  bar  when  the  answer 
is  correct.  Click  here  if  you  require  a  text-based  version  of  this  challenge. 
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Module  Summary  -  Recaps  essential  information  to  reinforce  attainment  of  the 
learning  objectives  of  each  lesson 
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Module  Summary 


Module  Summary  Organization 


The  module  summary  is  organized  by  lesson, 
It  presents  the  following  for  each  lesson: 

*  Learning  goal(s)  of  the  lesson 

*  Summary  of  the  key  learning  points 
needed  to  attain  each  learning  goal 


The  Module  Summary  includes  only  the 
following  lessons: 

*  System  Safety  Overview 

*  System  Safety  Terminology 

*  Eight  Mandatory  Steps  of  System 
Safety 

*  Risk  Assessment 

*  System  Safety  Order  of  Precedence 

*  Typical  System  Safety  Tasks 

*  System  Safety  Throughout  the 
System's  Life  Cycle 
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Conclusion 


■  Continuous  Learning  Course  helps  students 

■  Recognize  the  Defense  Acquisition  policy  and  guidance  on 
System  Safety  in  Systems  Engineering 

■  Recognize  System  Safety  as  the  Systems  Engineering  approach 
for  eliminating  ESOH  hazards  or  minimizing  ESOH  risks  across 
the  system  life  cycle 

■  Course  (CLE009)  available  for  registration  at  DAU’s  website 
http://www.dau.mil/basedocs/continuouslearning.asp 


25 


